High-quality colonoscopy is essential to the management of gastrointestinal diseases including the diagnosis and prevention of colorectal cancer (CRC). Poor-quality colonoscopy is associated with postcolonoscopy CRC (1–3) (PCCRC) and procedural complications (4,5), thereby increasing patient morbidity and mortality. There is significant variation in the quality of colonoscopy in Europe (6–9), the United States (1,10,11), and Canada (3,4,12,13). Observed variations in practice may signal underlying quality issues (14). Although the observed variation in the quality of colonoscopy is probably multifactorial, differences in endoscopist performance likely play an important role.
A/F is defined as ‟any summary of clinical performance of healthcare over a specified period of time” (15). In systematic reviews, A/F leads to a modest improvement in provider performance (15,16); however, these reviews have generally excluded trials where the goal was to improve procedural skills (15,16). Most published studies of A/F for colonoscopy have been small, single-center, uncontrolled studies which are susceptible to bias because of secular trends (17,18). Furthermore, it is not clear whether the A/F interventions in these trials were developed using established behavioral science principles. In controlled trials to date that incorporate A/F for colonoscopy, A/F typically comprises 1 element of a complex intervention—A/F alone has not been compared with a usual care arm (8,19,20).
Despite the weak evidence to support its use and the challenges in implementing effective A/F for colonoscopy (because of complexity related to the motor and cognitive components) (21), A/F has been widely implemented for colonoscopy in North America and Europe (22–26), likely in a highly variable fashion (25). We conducted a pragmatic randomized controlled trial to determine whether A/F for colonoscopy improves performance compared with usual practice. Because A/F may be more effective for those with low baseline performance (15,27), we also measured the effect of receiving A/F reports on colonoscopy performance among lower performing endoscopists.
MATERIALS AND METHODS
This study was undertaken with the assistance of ICES (formerly known as the Institute for Clinical Evaluative Sciences), the Quality Management Partnership (24), and Cancer Care Ontario (CCO). ICES is an independent, non-profit research institute whose legal status under Ontario's health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. The institutional review board at Sunnybrook Health Sciences Centre, Toronto, Canada, and the privacy offices at ICES and CCO approved the study (ClinicalTrials.gov: NCT02595775).
Health administrative databases held at ICES and CCO from Ontario's single payor health system, which services the 14 million residents of Ontario, were used to generate individualized A/F reports (see Supplementary Table S1, https://links.lww.com/AJG/C205).
We included all endoscopists who performed at least 5 colonoscopies in Ontario, Canada, in the calendar year 2014, regardless of specialty.
Endoscopists were randomly assigned 1:1 to either (i) receive A/F report and a resource sheet (intervention group) or (ii) not receive these materials (control group). A CCO analyst who was not involved in analyzing study data performed the randomization using a computer-generated randomization algorithm implemented in SAS software. Informed consent was not obtained from endoscopists to avoid selection bias. Study investigators, the ICES analyst who performed study analyses, CCO, and Quality Management Partnership policy-makers were blinded to endoscopist identity and allocation.
A/F reports and accompanying materials
As is recommended, A/F report and material development was guided by theory (21,28). The reports were generated centrally at ICES and comprised 9 evidence-based colonoscopy quality indicators (29) derived from health administrative data (Table 1, for indicators and definitions) for the period January 1, 2014, to December 31, 2014. Health administrative data in Ontario have been validated to measure important colonoscopy data elements (30).
The report was 1 double-sided page, describing the endoscopist's own performance using the 9 quality indicators, along with provincial averages and the endoscopist's rank relative to others on one side and indicator definitions on the other side (Figure 1). Previously, we tested a preliminary version of the A/F report with endoscopists for clarity, content, usability, and acceptability (31). A behavioral scientist (DL) and a data visualization expert revised the report for the trial to ensure that performance strengths and weaknesses could be easily identified by endoscopists when reading their report (32) and to incorporate findings from interviews.
Individualized A/F reports were couriered to endoscopists in the intervention arm on October 29, 2015 (index date). Before receiving their personal A/F report, these endoscopists were mailed a notification letter, indicating that the report was to be mailed within 2 weeks and highlighting key features of the report (e.g. it is confidential, informed by high-quality data, developed with input from their peers) intended to address anticipated concerns and to motivate them to use the report.
Along with the A/F report, physicians received a cover letter which was developed using principles of persuasive communication with the intention of motivating them to review the report (33) and a list of resources and incentives (i.e., eligibility for continuing medical education or available free/at a discounted price) to help them improve their colonoscopy practice (see Supplementary Appendix, https://links.lww.com/AJG/C204).
Colonoscopy performance was measured in the intervention and control groups over two 12-month periods: prereport and postreport (relative to the index date). The primary outcome was polypectomy rate (PR) (Table 1 for definition) because it has been shown to be associated with PCCRCs (3), because it can be measured in a timely fashion (unlike PCCRC), because there is known important variation in PR among endoscopists (13,34), and because it is a surrogate measure of the adenoma detection rate (ADR) (34,35). Improvement in ADR is associated with a reduction in risk of death from CRC (8,36).
Because adenoma pathology data were not available for the entire study period, ADR could not be used as the primary outcome. However, we performed a post hoc analysis using ADR. We have shown that natural language processing can be used to reliably identify adenomas using the noncancer pathology data held at CCO (37). We linked these data (available from June 1, 2015, onward) to the colonoscopies performed by the endoscopists in our study population and calculated each endoscopist's ADR. ADR was defined as the percent of colonoscopies where 1 or more adenomas were removed in persons aged 50 years and older. Because these data were not available for the entire prereport period, we restricted this period for this post hoc analysis to the 5 months before the index date. The postreport period was 12 months after the index date. ADR was defined as the proportion of outpatient colonoscopies in which ≥1 adenoma(s) were removed among persons aged 50 years and older.
Secondary outcomes are defined in Table 1 and included cecal intubation rate (CIR), poor bowel preparation, and proportion of colonoscopies with premature repeat after normal colonoscopy (an appropriateness measure). We did not measure changes in the other 5 indicators included in the A/F report (Table 1 for definitions) because they were not amenable to quality improvement efforts (e.g., annual colonoscopy volume) or they captured infrequent events (e.g., perforation) (serious adverse event counts are reported in Supplementary Digital Content; see Supplementary Table S7, https://links.lww.com/AJG/C211).
Endoscopist and patient factors
Using health administrative data, endoscopists were characterized by sex, specialty (gastroenterologist, surgeon, internal medicine, and other), years in practice (5–15 years, >15–25 years, >25–35 years, and >35 years), and practice setting (hospital-based, private clinic-based, or mixed, defined as more than 10 colonoscopies performed in each setting) (38). Patients were characterized by sex, age group (18–50 years, 51–74 years, and >75 years), median neighborhood income quintile (classified as rural or by urban income quintile) (39) as a proxy for patient socioeconomic status, and previous abdominal or gynecologic surgery.
Our principal analyses considered all endoscopists in the province who completed 6 or more colonoscopies (required by ICES Privacy Office to avoid small cell sizes) in the prereport and postreport periods. We also performed a prespecified subgroup analysis in lower performing endoscopists because A/F is suggested to be more effective in those with low baseline performance (15). Lower performing endoscopists were those with PR less than or equal to 25 percent, those with CIR less than or equal to 95 percent, and those in the highest quartile for poor bowel preparation and premature repeat after normal colonoscopy. Because ADR was not included in the A/F report but PR was, we used the PR definition for lower performing endoscopists in the ADR analyses because it is felt to be a surrogate for ADR.
Median and interquartile ranges for patient and endoscopist factors as of the index date as well as the prereport values for endoscopist PR, CIR, poor bowel preparation, and premature repeat after normal colonoscopy are reported. Standardized differences were used to compare these factors in the intervention and control groups in the prereport period. Variables where the standardized differences exceeded 0.1 were considered to differ importantly between the groups and were incorporated as covariates in the models described below.
Based on a difference-in-difference framework, primary and secondary outcomes were compared between the intervention and control groups by assessing both prereport and postreport periods using crude analyses and adjusted Poisson regression analyses. Crude analyses consisted of creating 2 × 2 tables of crude mean rates for each outcome by intervention group and by period; these rates were then used to compute the crude difference in difference estimate (40). For the physician-level adjusted Poisson regression analyses, the model consisted of the main intervention indicator, the period indicator, and also included an interaction term between the intervention and period indicators to determine whether the relative rate (RR) for the outcome (comparing the intervention against the control groups) was different in the prereport period compared with the postreport period (41). The offset term in the regression model was the natural logarithm of the number of patients belonging to each physician. These analyses were conducted overall for all endoscopists and separately for lower performing endoscopists. The post hoc analysis (called the ADR analysis below) was performed in the same manner as the other analyses except that the prereport period was limited to the 5 months before the index date. Finally, in other post hoc exploratory analyses, we evaluated the effect of A/F by endoscopist specialty and by endoscopist annual volume by repeating the main analysis separately for gastroenterologists and for general surgeons as well as for endoscopists in the lowest quartile and those in the upper 3 quartiles of annual colonoscopy volume. We also explored the durability of the A/F intervention by comparing ADR and PR in the first 6 months of the postreport period to the second 6 months of the postreport period. All tests were 2-sided, and P values of <0.05 were considered significant.
We estimated that 800 endoscopists would be practicing in Ontario in 2015. Assuming an average PR in the control group of 28% (13), there was 85% power to detect at least a 10% absolute increase in PR among physicians receiving the provider A/F report vs physicians who did not. A 10% absolute increase was felt to be a reasonable detectable difference because a randomized trial showed an 11% increase in ADR with A/F plus 2 hours of training (19).
Role of the funding source
The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
In 2014, 907 endoscopists were eligible and 881 were randomized (24 were no longer practicing in Ontario, and 2 were study investigators [J.T. and N.B.]) to the intervention and control groups. For the analysis, 24 were excluded in each group because they had not performed 6 or more endoscopies in each of the 2 periods, leaving 417 and 416 in the intervention and control groups, respectively. Among all physicians, a third were gastroenterologists, and more than half were surgeons, 80% were male, and the median annual colonoscopy volume was more than 400. Physician and patient characteristics were well balanced across these groups in the prereport period. However, among lower performing endoscopists in the PR analysis, baseline endoscopist characteristics differed by specialty, years in practice, and practice setting while patient characteristics were well balanced (Tables 2 and 3). Lower performing endoscopists in the CIR, bowel preparation, premature repeat after normal colonoscopy, and ADR analyses were balanced in terms of patient characteristics with some differences at the endoscopist level (see Supplementary Tables S2–S4, https://links.lww.com/AJG/C206, https://links.lww.com/AJG/C207, https://links.lww.com/AJG/C208).
Among all endoscopists, mean PR improved from the prereport to the postreport periods, without a significant difference between the intervention (from 39.9% to 42.4%) and control (from 40.0% to 41.8%) groups (RR, intervention vs control: 1.07 vs 1.05, P = 0.09). Among lower performing endoscopists, there was improvement in PR in both groups from the prereport to the postreport periods with significantly greater improvement in the intervention (from 17.9% to 23.8%) compared with the control (from 19.4% to 23.3%) group (RR, intervention vs control: 1.34 vs 1.11, P = 0.02) (Table 4). The 20% increase in PR (1.34/1.11) was the largest in magnitude compared with other outcomes for low performing endoscopists.
Among all endoscopists, there was a similar improvement in mean ADR from the prereport to the postreport periods in both the intervention and control groups (RR, intervention vs control: 1.03 vs 1.03, P = 0.83). Among lower performing endoscopists, the mean ADR improved to a greater extent in the intervention (from 18.6% to 20.3%) compared with the control (from 19.2% to 19.6%) group; however, the difference was not significant (RR, intervention vs control: 1.12 vs 1.04, P = 0.12) (Table 4).
Among all endoscopists and lower performing endoscopists, CIR, poor bowel preparation rate, and premature repeat after normal colonoscopy rate improved in both the intervention and control groups from the prereport to the postreport periods, without a significant difference between the 2 groups (Table 4).
We did not find differences in A/F effectiveness by specialty or endoscopist annual volume nor did we note a drop-off in ADR or PR from the first 6 months to the second 6 months postreport among all endoscopists or lower performing endoscopists (see Supplementary Tables S5, S6, https://links.lww.com/AJG/C209, https://links.lww.com/AJG/C210).
In this large pragmatic controlled trial, compared with no report, A/F reports for colonoscopy led to a significant improvement in PR and a nonsignificant improvement in ADR among lower performing endoscopists. Although there was improvement from the prereport to the postreport period in the other performance indicators among the lower performing endoscopists, and in all indicators among all endoscopists, this improvement did not differ significantly across the 2 randomized groups. This change cannot be attributed to A/F and likely reflects secular trends.
We observed that lower performing endoscopists derived the greatest benefit from A/F. Our findings were significant for PR only; however, the trends were similar for the ADR analysis, which may have been underpowered. These results are consistent with a systematic review that found that A/F is more effective when baseline physician performance is low (15). Others studying colonoscopy quality improvement have found most benefit in those with lowest performance at baseline (42) or have limited their study to low performers (8).
In randomized controlled trials to date that incorporated A/F for colonoscopy quality improvement, none have compared A/F alone with no report. Among these, there are 2 that reported a significant benefit from the addition of supplementary interventions to A/F, but neither trial included a no A/F report arm (US trial—2 hours of trainings on adenoma detection/polyp classification in all endoscopists at a single ambulatory center (19); Polish trial—a 3 days ‟hands-on” colonoscopy technical skills course in low performers across multiple centers in a colorectal cancer screening program (8)). A recent systematic review (18) suggested benefit from A/F alone for colonoscopy; however, the 12 included studies were small and largely single center that used a quasi-experimental design (i.e., compared post–A/F ADR with pre–A/F ADR). The lack of control group in these studies makes them vulnerable to bias; that is, the improvement in ADR may result from secular trends or a ‟rising tide” phenomenon (43) because of a heightened awareness among endoscopists of the importance of high-quality colonoscopy, rather than from the A/F intervention itself. Despite this weak evidence to date, A/F for colonoscopy is being used widely (22–26). Our rigorously designed, randomized controlled trial adds important information because it suggests that A/F alone may only benefit a subset of endoscopists.
In addition to its randomized controlled design, we designed our A/F intervention carefully considering its potential mechanisms of action as is recommended (28,44), which allowed us to address some of the challenges of using A/F for a complex procedure such as colonoscopy (17). We relied on Feedback Intervention Theory (21), developed for use in industry, which suggests that effectiveness of A/F is influenced by the characteristics of the feedback, the nature of the task requiring improvement, and the context, which includes the recipient's personality. In light of the complexity of both A/F and of colonoscopy, further gains in performance may be possible with a thoughtful multipronged approach to colonoscopy quality improvement that uses but is not limited to A/F alone. Pairing A/F with evidence-based quality improvement efforts, such as facilitated feedback (45), may be preferable to tying it to pay-for-performance, given the uncertain effectiveness of the latter (46).
Our A/F intervention incorporated evidence-based accepted quality indicators and was informed by A/F theory and literature; however, there were some limitations. Systematic reviews have shown that A/F is effective when provided in a timely, frequent manner (15). Despite the lag in reporting (related to using health administrative data for the reports) and that the report was only provided once, we found that A/F was beneficial in a subset of endoscopists, and although our exploratory analyses failed to show a drop-off in benefit from the first 6 months to the second 6 months postreport, repeated reports may be even more effective.
Using health administrative data, which are collected for nonresearch purposes, could lead some physicians to question their validity (31). However, there are advantages to using health administrative data: They are available for all practitioners in a health jurisdiction, they can be measured centrally, data on a single patient can be captured across institutions, and they can be inexpensively monitored at regular intervals. Some of these issues above may be addressed by additional interventions such as pairing the written report with facilitated feedback, an evidence-based approach to providing physician feedback using methods designed to enhance acceptance and use of the feedback (45).
Another potential issue is that we measured ADR and PR across colonoscopies for all indications, rather than in those undergoing primary screening colonoscopy, as originally described (47). However, recent colonoscopy quality assurance guidances have recommended measuring ADR across all colonoscopies (22,48); furthermore, we measured the incremental change in ADR and PR from baseline, which should address confounding by colonoscopy indication.
Finally, our ADR analyses were post hoc because pathology data only became available part way through the study. It is possible that the report would have had a greater impact on ADR had physicians received their ADR in the report.
A/F reports for colonoscopy improved PR among lower performing endoscopists. Although colonoscopy performance reporting is common, until now, there has been no randomized controlled trial evidence to support its use. Understanding the mechanisms of A/F may help to improve the way it is delivered. Further measures, such as multiple rounds of A/F reports and the use of supplementary quality improvement interventions, may provide additional benefit.
CONFLICTS OF INTEREST
Guarantor of the article: Jill Tinmouth, MD, PhD.
Specific author contributions: J.T., R.S., N.B., D.L., L.P., and L.R.: contributed to the conception and design of the study. J.T., R.S., Q.L., J.P., N.B., L.P., and L.R.: contributed to data analysis and data interpretation. J.T. and J.P.: drafted the work and all others contributed to revising it critically for important intellectual content. All authors reviewed, contributed to, and approved the final version of the manuscript.
Financial support: Canadian Cancer Society Research Institute. The funder was not involved in any aspect of the study.
Potential competing interests: J.T. is the Lead Scientist for the ColonCancerCheck program at CCO, at the time of the study, N.B. was the GI Endoscopy Lead at CCO, and L.R. is the Vice President, Prevention and Cancer Control at CCO. None of the other authors have any conflicts of interest to report.
WHAT IS KNOWN
- ✓ Evidence to support physician audit and feedback (A/F) for colonoscopy is limited to small, single-center uncontrolled studies, which are vulnerable to bias because of secular trends.
- ✓ Despite this limited evidence, A/F for colonoscopy is performed widely.
WHAT IS NEW HERE
- ✓ Performance improved for all indicators over time among those receiving A/F and those who did not, demonstrating the role of secular trends.
- ✓ A/F for colonoscopy, implemented across a large health system, led to improvements among lower performing endoscopists compared with usual care.
- ✓ Polypectomy rates improved with A/F for colonoscopy compared with usual care among lower performing endoscopists, but cecal intubation, bowel preparation, and appropriateness did not.
This study was conducted with the financial support of the Canadian Cancer Society Research Institute. This study was supported by Cancer Care Ontario (CCO) and by ICES; the latter is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results, and conclusions reported in this article are those of the authors and are independent from the above sources. No endorsement by CCO, ICES, or the Ontario MOHLTC is intended or should be inferred. Parts of this material are based on data and information compiled and provided by Canadian Institute for Health Information CIHI. However, the analyses, conclusions, opinions, and statements expressed herein are those of the authors and not necessarily those of CIHI. The authors also wish to acknowledge CCO analysts for their contribution to the study (Julia Gao, Dan Strumpf) and Dr. Karen Woods and the American Society of Gastrointestinal Endoscopy (ASGE) for providing us with 2 videos that were included among the educational resources offered to endoscopists in our trial.
1. Corley DA, Jensen CD, Marks AR, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014;370:1298–306.
2. Kaminski MF, Regula J, Kraszewska E, et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362:1795–803.
3. Baxter NN, Sutradhar R, Forbes SS, et al. Analysis of administrative data finds endoscopist quality measures associated with postcolonoscopy colorectal cancer. Gastroenterology 2011;140:65–72.
4. Rabeneck L, Paszat LF, Hilsden RJ, et al. Bleeding and perforation after outpatient colonoscopy and their risk factors in usual clinical practice. Gastroenterology 2008;135:1899–906.e1.
5. Laanani M, Coste J, Blotière P-O, et al. Patient, procedure, and endoscopist risk factors for perforation, bleeding, and splenic injury after colonoscopies. Clin Gastroenterol Hepatol 2018;17:719–27.e13.
6. Bowles CJ, Leicester R, Romaya C, et al. A prospective study of colonoscopy practice in the UK today: Are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277–83.
7. Gavin DR, Valori RM, Anderson JT, et al. The national colonoscopy audit: A nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013;62:242–9.
8. Kaminski MF, Anderson J, Valori R, et al. Leadership training to improve adenoma detection rate in screening colonoscopy: A randomised trial. Gut 2016;65:616–24.
9. Zorzi M, Senore C, Da Re F, et al. Quality of colonoscopy in an organised colorectal cancer screening programme with immunochemical faecal occult blood test: The EQuIPE study (evaluating quality indicators of the performance of endoscopy). Gut 2015;64:1389–96.
10. Shaukat A, Rector TS, Church TR, et al. Longer withdrawal time is associated with a reduced incidence of interval cancer after screening colonoscopy. Gastroenterology 2015;149:952–7.
11. Wernli KJ, Brenner AT, Rutter CM, et al. Risks associated with anesthesia services during colonoscopy. Gastroenterology 2016;150:888–94.e18.quiz e18.
12. Shah HA, Paszat LF, Saskin R, et al. Factors associated with incomplete colonoscopy: A population-based study. Gastroenterology 2007;132:2297–303.
13. Tinmouth J, Baxter NN, Li Q, et al. Measurement of an expanded suite of 9 colonoscopy quality indicators at the population level in Ontario, Canada. Gastrointest Endosc 2016;83:AB544–5.
14. Fisher ES, Wennberg DE, Stukel TA, et al. The implications of regional variations in Medicare spending. Part 2: Health outcomes and satisfaction with care. Ann Intern Med 2003;138:288–98.
15. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes cochrane. Database Syst Rev 2012;6:CD000259.
16. Hysong SJ. Meta-analysis: Audit & feedback features impact effectiveness on care quality. Med Care 2009;47:356.
17. Tinmouth J, Patel J, Hilsden RJ, et al. Audit and feedback interventions to improve endoscopist performance: Principles and effectiveness. Best Pract Res Clin Gastroenterol 2016;30:473–85.
18. Bishay K, Causada-Calo N, Scaffidi MA, et al. Associations between endoscopist feedback and improvements in colonoscopy quality indicators: A systematic review and meta-analysis. Gastrointest Endosc 2020;92:1030–40.e9.
19. Coe SG, Crook JE, Diehl NN, et al. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol 2013;108:219–27.
20. Wallace MB, Crook JE, Thomas CS, et al. Effect of an endoscopic quality improvement program on adenoma detection rates: A multicenter cluster-randomized controlled trial in a clinical practice setting (EQUIP-3). Gastrointest Endosc 2017;85:538–45.e4.
21. Kluger AN, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254–84.
22. Rees CJ, Thomas Gibson S, Rutter MD, et al. UK key performance indicators and quality assurance standards for colonoscopy. Gut 2016;65:1923–9.
23. Sadowski DC. Quality Reporting of Colonoscopy Performance Standards for the Alberta Colorectal Cancer Screening Program. Alberta Health Services: Alberta, Canada, 2013. (https://screeningforlife.ca/wp-content/uploads/2019/12/ACRCSP-Quality-Reporting-of-Colonoscopy-Performance-Standards-Feb-2013.pdf
). Accessed September 30, 2019.
24. Quality Management Partnership. Provincial Quality Management Programs for Colonoscopy, Mammography and Pathology in Ontario. Ontario, Canada, 2015. (https://qmp.ccohealth.ca/en
). Accessed March 30, 2018.
25. Gawron AJ, Lawrence P, Millar MM, et al. A nationwide survey and needs assessment of colonoscopy quality assurance programs in the VA. Fed Pract 2018;35:26–32.
26. Bronzwaer MES, Depla ACTM, van Lelyveld N, et al. Quality assurance of colonoscopy within the Dutch national colorectal cancer screening program. Gastrointest Endosc 2019;89:1–13.
27. Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006:CD000259.
28. Ivers NM, Sales A, Colquhoun H, et al. No more ‟business as usual” with audit and feedback interventions: Towards an agenda for a reinvigorated intervention. Implement Sci 2014;9:14.
29. Tinmouth J, Kennedy EB, Baron D, et al. Colonoscopy quality assurance in Ontario: Systematic review and clinical practice guideline. Can J Gastroenterol Hepatol 2014;28:251–74.
30. Tinmouth J, Sutradhar R, Liu N, et al. Validation of 5 key colonoscopy-related data elements from Ontario health administrative databases compared to the clinical record: A cross-sectional study. CMAJ open 2018;6:E330–E338.
31. Webster F, Patel J, Rice K, et al. How to make feedback more effective? Qualitative findings from pilot testing of an audit and feedback report for endoscopists. Can J Gastroenterol Hepatol 2016;2016:4983790.
32. Reyna VF. A theory of medical decision making and health: Fuzzy trace theory. Med Decis Making 2008;28:850–65.
33. Making Health Communication Programs Work: A Planner's Guide. US Department of Health and Human Services, Public Health Service, National Institutes of Health, [Office of Cancer Communications, National Cancer Institute], 2002 (https://www.cancer.gov/publications/health-communication/pink-book.pdf
34. Gohel TD, Burke CA, Lankaala P, et al. Polypectomy rate: A surrogate for adenoma detection rate varies by colon segment, gender, and endoscopist. Clin Gastroenterol Hepatol 2014;12:1137–42.
35. Williams JE, Holub JL, Faigel DO. Polypectomy rate is a valid quality measure for colonoscopy: Results from a national endoscopy database. Gastrointest Endosc 2012;75:576–82.
36. Kaminski MF, Wieszczy P, Rupinski M, et al. Increased rate of adenoma detection associates with reduced risk of colorectal cancer and death. Gastroenterology 2017;153:98–105.
37. Morgan D, Chorneyko K, Swain D, et al. A209 Validation of a natural language processing algorithm to extract data for system-level adenoma detection rate calculation. J Can Assoc Gastroenterol 2019;2:409.
38. Hol L, Sutradhar R, Gu S, et al. Repeat colonoscopy after a colonoscopy with a negative result in Ontario: A population-based cohort study. CMAJ open 2015;3:E244–50.
39. James PD, Wilkins R, Detsky AS, et al. Avoidable mortality by neighbourhood income in Canada: 25 years after the establishment of universal health insurance. J Epidemiol Community Health 2007;61:287–96.
40. Angrist JD, Pischke J-S. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press: Princeton, NJ, 2008.
41. Hanchate AD, Kapoor A, Katz JN, et al. Massachusetts health reform and disparities in joint replacement use: Difference in differences study. BMJ 2015;350:h440.
42. Gurudu SR, Boroff ES, Crowell MD, et al. Impact of feedback on adenoma detection rates: Outcomes of quality improvement program. J Gastroenterol Hepatol 2018;33:645–9.
43. Chen YF, Hemming K, Stevens AJ, et al. Secular trends and evaluation of complex interventions: The rising tide phenomenon. BMJ Qual Saf 2016;25:303–10.
44. Brehaut JC, Colquhoun HL, Eva KW, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med 2016;164:435–41.
45. Sargeant J, Lockyer J, Mann K, et al. Facilitated reflective performance feedback: Developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med 2015;90:1698–706.
46. Mendelson A, Kondo K, Damberg C, et al. The effects of pay-for-performance programs on health, health care use, and processes of care: A systematic review. Ann Intern Med 2017;166:341–53.
47. Rex DK, Petrini JL, Baron TH, et al. Quality indicators for colonoscopy. Gastrointest Endosc 2006;63:S16–28.
48. Kaminski MF, Thomas-Gibson S, Bugajski M, et al. Performance measures for lower gastrointestinal endoscopy: A European society of gastrointestinal endoscopy (ESGE) quality improvement initiative. Endoscopy 2017;49:378–97.