The paediatric series of direct observation of procedural skills (DOPS) were introduced into the UK national endoscopy training curriculum in 2016, but lack validity evidence. We aimed to present validity evidence for paediatric colonoscopy DOPS and study competency development in a national trainee cohort.
This prospective UK-wide study analysed formative paediatric colonoscopy DOPS which were submitted to the e-Portfolio between 2016 and 2018. Item, domain and average DOPS scores were correlated with the overall DOPS rating to evidence internal structure validity. Overall DOPS ratings were compared over lifetime procedure count to demonstrate learning curves (discriminant validity). Consequential validity was founded on receiver operating characteristic curve (ROC) analyses.
203 DOPS assessments were completed for 29 trainees from 11 UK training centres. Internal structure validity was provided through item-total correlation analyses. DOPS scores positively correlated with trainee seniority (P < 0.001) and lifetime procedure count (P < 0.001). Competency acquisition followed the order of: ‘pre-procedure’, ‘post-procedure’, ‘endoscopic non-technical skills’, ‘management’, ‘procedure’ domains, followed by overall DOPS competency, which was achieved in 81% of the cohort after 125–149 procedures. Mean DOPS scores could be used to predict overall procedure competence (area under ROC 0.969, P < 0.001), with a mean score of 3.9 demonstrating optimal sensitivity (93.5%) and specificity (87.6%).
This study provides validity evidence supporting the use of paediatric colonoscopy DOPS as an in-training competence assessment tool. DOPS may also be used to measure competency development and benchmark performance during training, which may be of value to trainees, trainers and training programmes.
*NIHR Biomedical Research Centre, University of Birmingham, Birmingham, UK
†Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK
‡Department of Paediatric Gastroenterology, Royal Free Hospital, London, UK
§Department of Gastroenterology, University Hospitals Birmingham, Birmingham, UK
¶Department of Paediatric Gastroenterology, John Radcliffe Hospital, Oxford, UK
||Department of Gastroenterology, Torbay and South Devon NHS Foundation Trust, Torquay, UK
**Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Cheltenham, UK
††Department of Paediatric Gastroenterology, Alder Hey Children's Hospital, Liverpool, UK
‡‡Department of Paediatric Gastroenterology, Sheffield Children's NHS Foundation Trust, Western Bank, Sheffield, UK
Address correspondence and reprint requests to Dr Keith Siau, Joint Advisory Group on Gastrointestinal Endoscopy, London, United Kingdom (. e-mail: firstname.lastname@example.org).
Received 28 October, 2018
Revised 20 December, 2018
Accepted 22 January, 2019
Competing Interests: KS is the JAG research fellow,
JA led in the development of adult colonoscopy DOPS,
PN led on the development and implementation of the paediatric colonoscopy DOPS
Author contributions: KS drafted the original manuscript and performed the statistical analyses. PN conceived the study and is the article guarantor. All authors provided critical review and enhanced the quality of the manuscript.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal's Web site (www.jpgn.org).