The paediatric series of direct observation of procedural skills (DOPS) were introduced into the UK national endoscopy training curriculum in 2016, but lack validity evidence. We aimed to present validity evidence for paediatric colonoscopy DOPS and study competency development in a national trainee cohort.
This prospective UK-wide study analysed formative paediatric colonoscopy DOPS which were submitted to the e-Portfolio between 2016 and 2018. Item, domain, and average DOPS scores were correlated with the overall DOPS rating to evidence internal structure validity. Overall DOPS ratings were compared over lifetime procedure count to demonstrate learning curves (discriminant validity). Consequential validity was founded on receiver operating characteristic curve analyses.
A total of 203 DOPS assessments were completed for 29 trainees from 11 UK training centres. Internal structure validity was provided through item-total correlation analyses. DOPS scores positively correlated with trainee seniority (P < 0.001) and lifetime procedure count (P < 0.001). Competency acquisition followed the order of: “preprocedure,” “postprocedure,” “endoscopic nontechnical skills,” “management,” “procedure” domains, followed by overall DOPS competency, which was achieved in 81% of the cohort after 125 to 149 procedures. Mean DOPS scores could be used to predict overall procedure competence (area under receiver operating characteristic curve 0.969, P < 0.001), with a mean score of 3.9 demonstrating optimal sensitivity (93.5%) and specificity (87.6%).
This study provides validity evidence supporting the use of paediatric colonoscopy DOPS as an in-training competence assessment tool. DOPS may also be used to measure competency development and benchmark performance during training, which may be of value to trainees, trainers, and training programmes.
*Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians
†Department of Paediatric Gastroenterology, Royal Free Hospital, London
‡Institute of Translational Medicine, Institute of Immunology and Immunotherapy and NIHR Birmingham Biomedical Research Centre, University Hospitals NHS Foundation Trust and University of Birmingham, Birmingham, UK
§Department of Paediatric Gastroenterology, John Radcliffe Hospital, Oxford
||Department of Gastroenterology, Torbay and South Devon NHS Foundation Trust, Torquay
¶Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Cheltenham
#Department of Paediatric Gastroenterology, Alder Hey Children's Hospital, Liverpool
**Department of Paediatric Gastroenterology, Sheffield Children's NHS Foundation Trust, Western Bank, Sheffield, UK.
Address correspondence and reprint requests to Dr Keith Siau, Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, 11 St Andrew's Place, London, NW1 4LE, UK (e-mail: email@example.com).
Received 28 October, 2018
Accepted 22 January, 2019
Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal's Web site (www.jpgn.org).
K.S. is a JAG research fellow. The remaining authors report no conflicts of interest.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Website (www.jpgn.org).