There is consensus that training in minimally invasive surgery should be intensified and that the assessment of the surgeons’ skills should be introduced to ensure high-quality treatment.1,2 Continued advances in computer technology combined with the growing need for training in advanced laparoscopic skills outside the operating room, have led to exponential growth and development of a variety of medically oriented virtual reality (VR) simulators.3–8 The use of simulation in surgical training curricula is becoming more widely accepted, for several reasons.2–5 First, simulator training is known to enhance the acquisition of laparoscopic skills.1,2,9–13 Second, validated VR simulators are able to provide objective assessment and feedback of all the subjects’ performance,2,5 allowing for continuous skill refinement. And finally, 2 randomized, controlled, double-blinded clinical trails have demonstrated that the clinical performance of subject receiving VR training is better than that of control subjects, who received no prior training.1,14 Objective assessment of the performance is fundamental to provide formative feedback during training, though current laparoscopic video training is lacking this ability. To assess the performance on the traditional box trainers, a scoring system has been developed in the form of the Fundamentals in Laparoscopic Surgery.15 However, an expert observer is still needed to assess the performance of the trainee. The ProMIS augmented reality (AR) laparoscopic simulator retains the benefit of a video trainer, such as the realistic haptic feedback, by using the same laparoscopic instruments as in the clinical setting and tactile tasks, and additionally generates objective measures of performance, similar to VR simulators.16 “Haptic feedback” is the phenomenon of tactile or force feedback a person experiences when manipulating a needle and thread. This is of importance, because resistance of the instruments and force/torque applied tissue also pertain to “haptics.”
Augmented Reality is the combination of physical and VR simulation in 1 system. The laparoscopic instruments are tracked by the system to measure the performance of each task. This results in the objective assessment of the real physical tasks performed by the trainees.
As is apparent from the European Association of Endoscopic Surgeons consensus guidelines from Carter et al,5 there is a need for validation of the assessments of the simulators, before including them into training curricula.2 The face validity addresses the acceptance of both experts and nonexperts (referent group) of the simulator as a potent training tool and the representation of the clinical setting. In this study, the face validity of the ProMIS AR laparoscopic simulator is investigated.
MATERIALS AND METHODS
The participants were tested within a period of 2 months (May–June 2006) at the “Year Congress of Dutch Surgical Society 2006,” Veldhoven, the Netherlands (N = 41), and the Catharina Hospital, Eindhoven (N = 14). The participants were allotted to 2 groups based on their laparoscopic experience: “Experienced,” who have done more than 50 laparoscopic procedures and at least 5 laparoscopic sutures in the clinical setting; and “Moderately experienced,” who have performed less than 50 procedures and less than 5 sutures.
The questionnaire used in this study (Appendix) was based on questionnaires previously used in validation studies in our research center,17,18 and consisted of 2 parts. The first part dealt with the demographics and laparoscopic (simulator) experience of participants. The second part consisted of multiple questions concerning the realism, didactic value, and haptic feedback of the simulator.
All questions were rated on a 5-point Likert scale, in which 1 stood for not true/realistic/useful and 5 for very true/realistic/useful, 3 was considered neutral.
In this study, the ProMIS AR simulator (Haptica, Dublin, Ireland) (Fig. 1) was used and validated. The laparoscopic interface consists of a torso-shaped mannequin (29 in. L × 20 in. W × 9 in. D), with a black neoprene cover, connected to a notebook with a standard 4-pin 1394 IEEE digital cable. The mannequin contains 3 separate camera tracking systems, arranged to identify any instrument inside the simulator from 3 different angles. The camera tracking systems capture instrument motion with Cartesian coordinates in the x, y, and z planes at the average rate of 30 frames per second (fps). The distal end of the laparoscopic instrument shaft is covered with 2 pieces of yellow electrical tape to serve as a reference point for the camera tracking system; therefore, it accepts a broad range of instrument types. Instrument movement is recorded and stored in distinct sections, based on the time the tips of the instrument are detected until they are removed from the mannequin. The data from the simulator is processed using a portable notebook computer (Sony Vaio, 2.80-GHz Intel Pentium 4 processor, running Windows XP Home Edition with 512 MB RAM and a 30 GB hard drive). The notebook was positioned so that the participant had the screen placed just below eye level, and the mannequin was placed at a standard ergonomic height for performing the laparoscopic tasks.
The simulator records “time,” “path length,” and “smoothness of movement” (through changes in instrument velocity and changes in direction) during each separate task within the training module. After completion of the task, ProMIS provides statistics on the screen. In addition, a full video and virtual playback of the trainee’s performance are saved. Different trays may be placed in the mannequin for each task, such as the suturing pads for the suture and knot tying task. For the translocation task, the 5-mm endograsps (Tyco Auto Suture, New Haven, CT) were used, and for the suturing tasks, the needle-holders (Karl Storz, Tutlingen Germany) with Tyco Polysorb 3–0 suturing needle and thread were used (Fig. 2). The sutures were placed on a 1-cm thick suturing pad, which is normally used in traditional box trainers.
The participants commenced the study by filling out the first part of the questionnaire. Subsequently, all participants received the same introduction of the simulator, with the aid of an informative poster. Information was given about the various training modules available and feedback provided by the simulators, as well as the tasks to be performed within the scope of this study. The simulator displayed a demonstration video before the task and step-by-step explanations were given during the performance. Next, the participants performed the translocation task (Fig. 3), followed by the “suturing and knot-tying” task (Fig. 4). After finishing both tasks, the participants completed the remaining part of the questionnaire, regarding their opinion on the simulator. A maximum time limit of 5 minutes was allotted for the translocation task, and 10 minutes were allotted for the suturing and knot-tying task (5 minutes for placing the suture and 5 for tying the knot), because surgical residents are generally expected to be able to perform such a task easily within this time.
The data were processed and analyzed with the Statistical Package for the Social Sciences version 13.0 using parametric tests. Data on the difference of opinion between the 2 experience groups were analyzed with the independent t test. A level of P ≤ 0.05 was considered statistically significant.
Demographics and Experience
A total of 55 subjects participated in this study, of which 27 were “Experienced” and 28 were “Moderately experienced.” The groups were not homogenous in all demographic aspects (Table 1). Both groups contained relatively more male participants (92.6% respectively 85.7%). Most of the participants were right-handed (96.7%). The age differences were also considerably between the groups (Table 1).
Issues about the clinical and nonclinical laparoscopic experience are stated in Table 2. The moderately experienced participants had the most experience on laparoscopic simulators, and may be explained by the fact that simulators have recently started to be employed in the training curricula of surgical residents. The training environments used by some of the participants were VR simulators and inanimate box trainers. The ProMIS simulator investigated in this study was only previously tried by 7 participants.
There was no significant difference found in the opinion between the 2 groups concerning the questioned features of the tasks of the ProMIS laparoscopic simulator (Table 3). The realism of the ProMIS was considered good to excellent by all participants, especially regarding the needle and thread. From the experienced 81.5% considered the ProMIS as a good to perfect representation (scoring 4 and 5 on the 5-point Likert scale) of the clinical setting. The haptic sensations during manipulating of the tissue were regarded as less realistic than the other parameters related to suturing by all participants, but was still regarded as good to excellent by 69.0% of the participants. Of all participants 82.8% regarded the resistance of the needle and thread as realistic to extremely realistic.
Regarding the value of the simulator, differences in opinion, between the 2 groups, were found on the properties of the demonstration videos (P = 0.004, P = 0.055, respectively). As outlined in Table 4, the ratings of the moderately experienced participants were lower on these features, whereas the experienced were more satisfied.
The ProMIS surgical simulator scored high as a training tool for the training of surgical residents (Table 5), with little variance in opinion. Of the experienced, 66.7% were of the opinion that the ProMIS AR simulator was excellent for training laparoscopic skills to surgical residents, and half of them thought the simulator to be very useful for the training of surgical specialists.
Augmented Reality Features
The simulation of surgical operations, particularly for VR, is very complex, especially when referring to laparoscopic techniques. VR simulators with enough computing power, tailored hardware and software programs to perform (fairly) realistic simulations have become available only for the past few years. Several studies have been carried out on different surgical simulators to examine their training capacities.19–22 For learning new skills it is important that trainees receive demonstrations, explanation, and feedback. This is a feature VR simulators offer in addition to those provided by traditional box trainers. The ProMIS AR simulator contains these VR features, while retaining the benefits of a realistic environment. Multiple studies3,8,9,19 have shown that haptic feedback is a valuable feature for good laparoscopic training. The ProMIS AR simulator offers a physically realistic training environment that is based on real instruments interacting with real objects, creating realistic haptic feedback, which is absent in VR simulators.
The realism and haptic feedback are important features that distinguish AR from the VR simulators. The participants from the current study considered these aspects of the ProMIS AR simulator as good to excellent, in particular for the suturing module. The experienced participants rated the realism with a mean of 4.37 on the 5-point Likert scale, which indicates a very realistic representation of clinical suturing and knot tying skills. The suturing material was regarded as less realistic, but this can be adjusted to individual preference for this simulator. For this study, we used suturing pads, which have been successfully used in box trainers at our skills laboratory for several years. However, not much is known about the realism of the haptic feedback they provide. In some VR simulators, the virtual instruments change as needed for the procedure, but the material is fixed in the software. The aspect of a realistic environment and haptic feedback is still questionable in VR simulators.23 Overall, all participants regarded the ProMIS laparoscopic simulator as a good to excellent representation of the clinical setting, by both the experienced and the less experienced participants. This establishes the face validity of this simulator.
Inexperienced trainees need explanation and demonstration videos on the procedure to be performed. In contrast, more skilled trainees know the procedures, and only need the physical practice. They often considered the step-by-step explanation, which could block the view of the instruments as a nuisance. The experienced, however, were of the opinion that these explanations were a good teaching method for inexperienced trainees. In this study, all participants regarded the ProMIS AR laparoscopic simulator as an excellent training tool for the main target group (surgical residents).
The ProMIS AR simulator gave several measurements as feedback to assess the performance: “Time,” “Smoothness” and “Path length,” which were shown on the screen at the end of the performance. It was tempting for the participants to focus on the time instead of concentrating on the other measurements shown on the screen. “Time” as a sole measurement might not be the best criterion to grade the trainee on. The primary issue is that the correct technique is used, whereas time is a secondary issue.
Studies concerning construct validity have been performed on the ProMIS laparoscopic simulator using several modules, but in particular the suturing tasks.7 From these studies, we know that the ProMIS can determine the level of skills of the participants, by means of the parameters measured. This is an important aspect for implementing the ProMIS AR simulator as a potent training tool in the current training curricula. Next, it is a valuable tool for gaining and improving laparoscopic skills by interval practicing, once a certain level of skill is obtained.
There were some limitations to this study. The major part of the participants participated on a voluntary basis during a congress. The study took place in a separate room, and the researchers made sure there were as few observers as possible, to avoid bias in both the performance and the opinion of the participants. The time to practice on the simulator was limited, and in total, 30 minutes were spent on the simulator on average. This was including the explanations, demonstrations videos, and performing the tasks. If a participant was of the opinion that a question could not be answered, based on the lack of experience on the simulator, they received the opportunity to gain the appropriate experience to form their opinion.
The limitations of this AR simulator are the measurements used to base the assessment of the performance on “Time,” “Pathlength,” and “Smoothness.” They do give an impression of the skills level of the trainee,6 but they do not provide formative feedback on how to improve the skills and which part of the task or procedure has to be practiced more extensively.
Conclusion and Recommendation
In this study, we established the face validity of the ProMIS AR laparoscopic simulator for the basic skills, by means of the translocation task and the suturing and knot-tying skills. All-over ratings about ProMIS’s realism was favorable and uniform among both in the teaching group (experienced surgeons) as the user group (surgical residents in training). ProMIS is therefore considered as a valuable tool in training laparoscopic skills to both surgical residents and surgeons.
1.Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trail of virtual reality simulation
for laparoscopic skills training. Brit J Surg
2.Schijven MP. Virtual Reality Simulation
for Laparoscopic Cholecystectomy: The Process of Validation and Implementation in the Surgical Curriculum Outlined. Rotterdam: Optima Grafische Communicatie; 2005.
3.Jakimowicz JJ, Cuschieri A. Time for evidence-based minimal access surgery training: simulate or sink. Surg. Endosc.
4.Roberts KE, Bell RL, Duffy J. Evolution of surgical skills training. World J Gastroenterol
5.Carter FJ, Schijven MP, Aggerwal R, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc
6.Fichera A, Prachand V, Kives S, et al. Physical reality simulation
for training of laparoscopists in the 21st century. A multispecialty, multi-institutional study. JSLS
7.Sickle Van KR. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc
8.Sokollik C, Gross J, Buess G. New model for skills assessment and training progress in minimally invasive surgery
. Surg Endosc
9.Aggerwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg
10.Gallagher AG, Cates CU. Virtual reality training for the operating room and cardiac catheterisation laboratory. Lancet.
11.Satava RM. Accomplishments and challenges of surgical simulation
. Surg Endosc.
12.Torkington J, Smith SG, Rees BI, et al. The role of simulation
in surgical training. Ann R Coll Surg Engl.
13.Aggarwal R, Grantcharov TP, Eriksen JR, et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg.
14.Seymour N, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg
15.Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation
in laparoscopic surgery. Ann Surg
16.Duffy AJ, Hogle NJ, McCarthy H, et al. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc.
17.Schijven MP, Jakimowicz JJ. Face-, expert- and referent validity of the Xitact LS500 laparoscopy
simulator. Surg Endosc
18.Koch AD, Buzink SN, Heemskerk J, et al. Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy. Surg Endosc
19.Hance J. Evaluation of a laparoscopic video trainer with in-built measures of performance. JSLS
20.Ro CY, Toumpoulis IK, Ashton RC Jr, et al. The LapSim: a learning environment for both experts and novices. Stud Health Technol Inform
21.Tomulescu V, Popescu I. The use of LapSim virtual reality simulator in the evaluation of laparoscopic surgery skill. Preliminary results. Chirurgia (Bucur)
22.Uchal M, Raftopoulos Y, Tjugum J, et al. Validation of a six-task simulation
model in minimally invasive surgery
. Surg Endosc. 2005;19:109–116.
23.Madan AK, Frantzides CT, Tebbit C, et al. Participants’ opinions of laparoscopic training devices after a basic laparoscopic training course. Am J Surg
Keywords:© 2008 Society for Simulation in Healthcare
Laparoscopy; Minimally invasive surgery; Simulation; Augmented reality