Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Surgical Quality Assurance in COLOR III

Standardization and Competency Assessment in a Randomized Controlled Trial

Tsai, Alice Y-C. MSc, MRCS*; Mavroveli, Stella PhD*; Miskovic, Danilo PhD, FRCS*,||; van Oostendorp, Stefan MD‡‡‡; Adamina, Michel MSc, EMBA, HSG; Hompes, Roel MD, PhD; Aigner, Felix MBA, FEBS, FACS§; Spinelli, Antonino MD, PhD, FASCRS; Warusavitarne, Janindra PhD, FRACS||; Knol, Joep MD**; Albert, Matthew MD, FACS, FASCRS††; Nassif, George JR DO, FACS††; Bemelman, Willem MD, PhD; Boni, Luigi MD, FACS‡‡; Ovesen, Henrik MD§§; Austin, Ralph MS, FRCS¶¶; Muratore, Andrea MD||||; Seitinger, Gerald MD***; Sietses, Colin MD, PhD§§§; Lacy, Antonio M. MD, PhD†††; Tuynman, Jurriaan B. MD, PhD‡‡‡; Bonjer, H. Jaap MD, PhD‡‡‡; Hanna, George B. MD, PhD, FRCS*

doi: 10.1097/SLA.0000000000003537

Objective: The aim of this study was to develop an objective and reliable surgical quality assurance system (SQA) for COLOR III, an international multicenter randomized controlled trial (RCT) comparing transanal total mesorectal excision (TaTME) with laparoscopic approach for rectal cancer.

Background of Summary Data: SQA influences outcome measures in RCTs such as lymph nodes harvest, in-hospital mortality, and locoregional cancer recurrence. However, levels of SQA are variable.

Method: Hierarchical task analysis of TaTME was performed. A 4-round Delphi methodology was applied for standardization of TaTME steps. Semistructured interviews were conducted in round 1 to identify key steps and tasks, which were rated as mandatory, optional, or prohibited in rounds 2 to 4 using questionnaires. Competency assessment tool (CAT) was developed and its content validity was examined by expert surgeons. Twenty unedited videos were assessed to test reliability using generalizability theory.

Results: Eighty-three of 101 surgical tasks identified reached 70% agreement (26 mandatory, 56 optional, and 1 prohibited). An operative guide of standardized TaTME was created. CAT is matrix of 9 steps and 4 performance qualities: exposure, execution, adverse event, and end-product. The overall G-coefficient was 0.883. Inter-rater and interitem reliability were 0.883 and 0.986. To enter COLOR III, 2 unedited TaTME and 1 laparoscopic TME videos were submitted and assessed by 2 independent assessors using CAT.

Conclusion: We described an iterative approach to develop an objective SQA within multicenter RCT. This approach provided standardization, the development of reliable and valid CAT, and the criteria for trial entry and monitoring surgical performance during the trial.

*Department of Surgery and Cancer, Imperial College London, London, United kingdom

Department of Surgery, Kantonsspital Winterthur, Winterthur, Switzerland

Department of Surgery, Academic Medical Center Amsterdam, Amsterdam, Netherlands

§Department of Surgery, Campus Mitte, Charité, University Medicine, Berlin, Germany

Colon and Rectal Surgery Unit, Humanitas Research Hospital, Milan, Italy

||Depatment of Surgery, St Mark's Hospital, London, United kingdom

**Deoartment of Surgery, Jessa Hospital, Hasselt Belgium

††Center for Colon and Rectal Surgery, Florida Hospital Medical Group, Orlando, FL

‡‡Department of Surgery, Fondazione IRCCS - Ca’ Granda - Ospedale Maggiore Policlinico, University of Milan, Milan, Italy

§§Depatment of Surgery, Zealand University Hospital, Roskilde, Denmark

¶¶Department of Surgery, Colchester General Hospital, Colchester, United kingdom

||||Department of Surgical Oncology, Candiolo Cancer Institute, Candiolo, Italy

***Department of Surgery, Krankenhaus der Barmherzigen Brüder Graz, Graz, Austria

†††Department of Surgery, Hospital Clínic, Barcelona, Spain

‡‡‡Department of Surgery, VU Medical Center, Amsterdam, Netherlands

§§§Department of Surgery, Gelderse Vallei Hospital (Netherlands).

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (

The authors report no conflict of interests.

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.