Objective measurement of retention of laparoscopic skills: a prospective cohort study

Introduction: There has been an overall growth of 462% in laparoscopic procedures performed by surgical residents between 2000 and 2018. Therefore, training courses in laparoscopic surgery are advocated in many postgraduate programs. While the immediate effect is determined in some cases, the retention of acquired skills is rarely investigated. The objective of this study was to objectively measure the retention of laparoscopic technical skills to offer a more personalized training program. Methods: First year general surgery residents performed two fundamental laparoscopic skills tasks (Post and Sleeve and the ZigZag loop) on the Lapron box trainer. Assessment was performed before, directly after, and 4 months after completing the basic laparoscopy course. Force, motion, and time were the measured variables. Results: A total of 29 participants were included from 12 Dutch training hospitals and 174 trials were analyzed. The 4 months assessment of the Post and Sleeve showed a significant improvement in force (P=0.004), motion (P≤0.001), and time (P≤0.001) compared to the baseline assessment. The same was true for the ZigZag loop: force (P≤0.001), motion (P=0.005), and time (P≤0.001). Compared to the 4 months assessment, skill deterioration was present for the Post and Sleeve in the mean force (P=0.046), max impulse (P=0.12), and time (P=0.002). For the ZigZag loop, skill decay was observed for force (P=0.021), motion (P=0.015), and time (P≤0.001) parameters. Conclusion: Acquired laparoscopic technical skills decreased 4 months after the basic laparoscopy course. Compared to baseline performance, participants showed significant improvement, however deterioration was observed compared to postcourse measurements. To preserve acquired laparoscopic skills, it is recommended to incorporate maintenance training, preferably with objective parameters, in training curricula.


Introduction
The increase of minimally invasive surgery (MIS) is a growing trend across multiple countries and within multiple surgical specialties [1][2][3][4] . The implementation of MIS has been an essential development in surgery with many benefits such as: faster postoperative recovery, shorter hospital stay, less surgical trauma, and reduced scarring compared to traditional open surgery [5][6][7][8][9][10] . Furthermore, an increase in the demand for enhancing and training laparoscopic technical skills of surgeons and residents has also been observed [11][12][13] . Surgical residents had an increase of 462% in laparoscopic procedures between 2000 and 2018 [3] .
Training in MIS techniques such as laparoscopy has proven to reduce errors during surgery and to optimize patient safety and outcomes [14][15][16][17][18][19] . Regardless of training curricula methods such as virtual reality training, video training or laparoscopic box training, assessment of technical skill is also important. Assessment and feedback on performance offer trainees with a clear goal and the degree of skill progression also increases as trainees are more motivated [20][21][22] . Optimization of MIS training, in order to reduce costs of expensive facilities, requires knowledge on effect of training and retention of trained skills.

HIGHLIGHTS
• Objective assessment in laparoscopic training, showed skill decay after 4 months. • This is the first study to assess skill retention with force, motion, and time. • The trainees remain better compared to their baseline assessment. • Previous studies used subjective assessment.
• Maintenance laparoscopic simulation training using objective feedback is essential. Prior studies have investigated and observed surgical skill decay using subjective assessment methods and reported the importance of maintenance training [23,24] . However, no objective parameters have been priorly used to compare baseline with retention measurements. In other words, a research gap exists regarding defining laparoscopic skills retention and deterioration, using objective parameters such as force and tissue manipulation parameters.
Previously, our research team validated the basic laparoscopy course (BLC) (Amsterdam Skills Centre) using objective force, motion, and time parameters [22,25] . The BLC showed improvement of basic technical laparoscopic skills of surgical residents in an early phase of the learning curve. Next to this, studies have shown that maintenance training should be part of every simulator curriculum as surgical skill decay was observed using subjective assessment methods [23,24,[26][27][28] . The aim of this study was to analyze long-term MIS skill retention, based on objective parameters in surgical novices.

Study design and participants
In this multicenter prospective cohort study first year Dutch surgical residents (PGY 1) that completed the BLC were included for the analysis. Ethical Review Board approval was not required and participation was on a voluntary basis. The participants received a questionnaire (Supplemental File A, Supplemental Digital Content 1, http://links.lww.com/JS9/A167) for baseline characteristics, average time since the completion of the BLC and laparoscopic experience.

Protocol
The BLC consists of a 3 weeks at-home laparoscopic box training with seven previously validated laparoscopic tasks [22,29] . The BLC is followed by a training day 4-6 months after the course in which the participants perform a laparoscopic cholecystectomy and appendectomy on fix for life cadaver models [30] .
To assess retention of skill, data of two tasks of different complexity were analyzed (Fig. 1, Post and Sleeve and ZigZag Loop). Participants performed a baseline assessment (BLA) before the BLC, a postcourse assessment (PCA) after the BLC and a retention test at the 4-6 months training day (4MA) (Fig. 2). The objective parameters of the 4MA were compared to the BLA and the PCA. Study protocol and work was reported according to STROCSS criteria [31] , Supplemental Digital Content 2, http:// links.lww.com/JS9/A168.

Subanalysis of the 4MA
Analyses were performed to compare the effect of time between box training and the 4-month assessment. The participants were divided into three groups: 0-3, 4-6, and greater than 7 months. Furthermore, the effect of experience in laparoscopic procedures (0, 1-10, and > 10) and simulation training (0-150, 151-240, and > 241 min) on the 4MA was also analyzed.

Systems and hardware
The Lapron box trainer (Amsterdam Skills Centre, Amsterdam, The Netherlands) (File b, Supplemental File A, Supplemental Digital Content 1, http://links.lww.com/JS9/ A167) was used during the BLC. The Lapron box trainers were equipped with the ForceSense objective measuring system (MediShield B.V.) [32] . The box trainer provided in total seventeen force, motion, and time parameters of which the most relevant parameters are shown in Table A1 (Supplemental File A, Supplemental Digital Content 1, http:// links.lww.com/JS9/A167) [22,32,33] . The tasks were recorded and uploaded with the objective parameters to an online database. The Lapron box trainers were also equipped with two curved Maryland grasping forceps (Aesculap, B. Braun) for the laparoscopic tasks.

Statistical analyses
The objective parameters were imported from the online database into IBM SPSS statistics 28 (SPSS Inc.) for statistical analyses. The Shapiro-Wilk test was performed to test for normality and the data was not normally distributed. Nonparametric tests were chosen for statistical analyses. The Wilcoxon signed-rank test was conducted to compare the 4-month assessment with the baseline course assessment and PCA. Mann-Whitney U-test was performed to determine the effect of laparoscopic experience (number of procedures and hours of simulation training) and time since BLC completion, on the retainability of laparoscopic technical skills. Differences were considered statistically significant if P less than 0.05.   Four-month assessment compared to postcourse assessment Table 1 and Figure 3 show Also, the total path length (4572 vs. 3849 mm; P = 0.015), and time (76 vs. 60 s; P ≤ 0.001) were also higher.

Four-month assessment compared to baseline assessment
The results of the 4MA and BLA comparison are provided in Table 2 and

Effect of experience in laparoscopic procedures and simulation training
The effect of experience in laparoscopic procedures and simulation training on the 4MA, is presented in Table B5 and B6 (Supplemental File B, Supplemental Digital Content 1, http://links.lww.com/JS9/A167). No significant differences, based on laparoscopic experience, were observed in the objective outcome measurements of the 4MA. Laparoscopic simulation training also showed no significant difference in the 4MA results.

Effect of time between box training and retention test
The effect of time between the box training and 4MA are demonstrated in Table B4

Discussion
This study demonstrated that participants of the BLC show skill decay, on average after 4 months, for the Post and Sleeve task (3/ 11 parameters) and ZigZag loop (5/11 parameters) when compared to the PCA. Although, the laparoscopic technical skills of the trainees deteriorate compared to just after the course, the trainees remain better compared to their BLA. For the most difficult task (ZigZag loop), the participants even perform significantly better for all objective parameters (11/11) compared to the first measurement.
No effect was observed on the 4-month assessment regarding laparoscopic experience and time between box training and training day.
Participant performance after 4 months already showed an significant deterioration of laparoscopic technical skill, especially force measurements, and tissue handling parameters. However, this does not exclude that a trainee might have improved in procedural skills or knowledge. Surgical skills simulation training and assessment should not be limited to International Journal of Surgery one-off training courses but be an intrinsic part of the residents' and surgeons career and learning curve. It is important to define a clear preset proficiency level or expert bench mark to continuously analyze the learning curve of participants during the training years. Specifically, during low volume moments of the residency training program. Prior research has reported various outcomes regarding the retainability and decay of laparoscopic technical skills. Some studies showed skill retention 1-2 years after the initial measurement [26][27][28] . However, other studies have shown deterioration of laparoscopic technical skills 6 months to 1 year after the baseline test [34][35][36][37][38] .
All previously conducted studies used subjective assessment. The studies performed an initial measurement on a laparoscopy box or virtual reality trainer, which was followed up by a postmeasurement at various time intervals. Next to subjective assessment, laparoscopic skills were assessed by total completion time of the task or task specific errors. This does not take into account objective parameters such as force (that mimic tissue manipulation) and the movements of laparoscopic instruments.
A strong point of the current study is the objective measurement of force, motion, and time, avoiding subjective assessment/ evaluation by individual assessors. This results in no interrater variability and the assessment can be compared objectively. Also, this study had a relatively big and homogeneous inclusion of surgical residents (PGY 1) from multiple training hospitals in the Netherlands.
In conclusion, the basic laparoscopic course is efficient in acquiring and retaining fundamental laparoscopic technical skills and the improvement of skill is prevalent after 4 months. To maintain the laparoscopic skills, it is desirable to consistently practice laparoscopic skills with objective force, motion, and time feedback.

Ethical approval
The Basic Laparoscopy Course is part of the surgical residency program, and participating in the study was voluntary (and without consequences). The study was exempt from Ethical Board review. No patient data was used; no ethics approval was required, and the study has thus not been registered with patient research registries or databases.

Sources of funding
No sources of funding were used.

Author contribution
M.R.: conception and design, drafting the article, final approval, study design, inclusions, data collection and analysis, manuscript writing and revisions. S.F.H.: conception and design, drafting the article, final approval, study design, inclusions, data collection, manuscript writing and revisions. S.R.S.: conception and design, drafting the article, final approval, inclusions, data collection and analysis, manuscript writing. H.J.B. and F.D.: conception and design, drafting the article, final approval, study design, methods, manuscript writing, manuscript revisions, supervision.

Conflicts of interest disclosure
All authors declare that they do not have any conflicts of interest or financial ties to disclose.