Secondary Logo

Journal Logo

Technical and Nontechnical Skills in Surgery

A Simulated Operating Room Environment Study

Pfandler, Michael MSc; Stefan, Philipp Dipl.-Inf.(Univ.); Mehren, Christoph MD; Lazarovici, Marc MD§; Weigl, Matthias Dr.§

doi: 10.1097/BRS.0000000000003154
SURGERY
Free
SDC
Blog

Study Design. Observational simulation study.

Objective. The goal of this study was to investigate the relationship between technical and nontechnical skills (NTS) in a simulated surgical procedure.

Summary of Background Data. Although surgeons’ technical and NTS during surgery are crucial determinants for clinical outcomes, little literature is available in spine surgery. Moreover, evidence regarding how surgeons’ technical and NTS are related is limited.

Methods. A mixed-reality and full-scale simulated operating room environment was employed for the surgical team. Eleven surgeons performed the vertebroplasty procedure (VP). Technical skills (TS) were assessed using Objective Structured Assessment of Technical Skill scores and senior expert-evaluated VP outcome assessment. NTS were assessed with the Observational Teamwork Assessment for Surgery. Kendall-Tau-b tests were performed for correlations. We further controlled the influence of surgeons′ experience (based on professional tenure and number of previous VPs performed).

Result. Surgeons’ NTS correlated significantly with their technical performance (τ = 0.63; P = 0.006) and surgical outcome scores (τ = 0.60; P = 0.007). This association was attenuated when controlling for surgeons’ experience.

Conclusion. Our results suggest that spine surgeons with higher levels of TS also apply better communication, leadership, and coordination behaviors during the procedure. Yet, the role of surgeons′ experience needs further investigation for improving surgeons’ intraoperative performance during spine surgery.

Level of Evidence: 3

We conducted an observational study drawing upon a mixed-reality simulation of a vertebroplasty. Technical and nontechnical skills were assessed and showed significant correlations. The correlations were attenuated when they were controlled for surgeons′ experience.

Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig Maximilians University, Munich, Germany

Chair for Computer Aided Medical Procedures & Augmented Reality, Department of Informatics/I-16, Technical University of Munich, Munich, Germany

Spine Center Schön Clinic Munich Harlaching, Munich, Germany; Academic Teaching Hospital Paracelsus Medical University (PMU), Salzburg, Austria

§Institute for Emergency Medicine and Management in Medicine (INM), University Hospital, Ludwig Maximilians University, Munich, Germany.

Address correspondence and reprint requests to Michael Pfandler, MSc, Institute and Outpatient Clinic for Occupational, Social, and Environmental Medicine, University Hospital, Ludwig-Maximilians-University Munich, Ziemssenstrasse 1, 80336 Munich, Germany; E-mail: michael.pfandler@med.uni-muenchen.de; ORCID: 0000-0002-1262-7068

Received 1 March, 2019

Revised 23 May, 2019

Accepted 5 June, 2019

M.P. and P.S. are co-first authors and have contributed equally to this work.

The manuscript submitted does not contain information about medical device(s)/drug(s).

The German Research Foundation (DFG, WE 4682/4-1, NA 620/30-1, and LA 3341/2) and the Munich Centre for Health Sciences (MC-Health) funds were received in support of this work.

Relevant financial activities outside the submitted work: grants.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Website (www.spinejournal.com).

Adverse events in the operating room (OR) are often linked to failures in nontechnical skills (NTS).1,2 Although various definitions of NTS are available, the majority include intraoperative communication, situational awareness, decision-making, teamwork, and leadership.3,4 Specifically for orthopedic surgery, 44% of deaths had a failure in NTS with missing situational awareness accounting for the majority (51.7%) of reported incidents.5 Therefore, NTS are fundamental for procedure efficiency, patient outcomes, surgical success, and patient satisfaction.2,6 To the best of our knowledge, there are no studies on NTS in spine surgery available. Technical skills (TS), however, are defined as psychomotor actions or related mental faculties acquired through practice and learning pertaining to a particular craft or profession.7 Low levels of NTS, especially situational awareness, are associated with a higher likelihood of technical errors.3,8,9 Although NTS and TS are considered to be related, specific effects of NTS on patient outcomes as well as their interrelations to TS remain unclear.10 The current literature base on surgeon's experience and intraoperative NTS is inconsistent.11–13 Moreover, the role of surgical experience within the interplay of NTS and TS has not been surveyed yet.3,8,9 Simulation-based performance scenarios of surgical teams in realistic OR environments provide unique opportunities to scrutinize the interplay of surgeons’ TS and NTS during spine procedures.14 In other specialties, simulation studies revealed positive correlations between TS and NTS during intraoperative crisis resource management15 as well as open gastrojejunostomy16 and ureteroscopy.17

The purpose of this study was to investigate for associations between surgeons’ intraoperative NTS and TS and to determine the influence of surgeons’ experience on these associations.

Back to Top | Article Outline

MATERIALS AND METHODS

We followed the reporting guidelines for health care simulation research.18 Ethical approval was obtained from the Ethics Committee of the Faculty of Medicine, LMU University (Nr 773-15). All participants were informed about the purpose of the study, confidentiality of data, and provided written consent before data collection.

Back to Top | Article Outline

Study Design and Setting

We conducted an observational study with a simulated mixed-reality procedure of a vertebroplasty procedure (VP) in a full-scale and realistic OR environment. We recruited surgeons from two university teaching hospitals in Germany. We applied a convenience sampling approach through snowball invitations to all eligible surgeons within the respective departments. The study took place in the simulation center of the Institute for Emergency Medicine and Management in Medicine (INM). No changes were made to the study protocol after trial commencement.

Back to Top | Article Outline

Mixed-reality Surgical Simulation Environment

We simulated a two-sided VP of a fractured lumbar vertebra (L2). The setup allows for complete intraoperative workflow of a VP for a surgical team consisting of surgeon, anesthetist, scrub nurse and circulating nurse. The previous definition of procedural workflow, specification of role demands, and requirements for simulation environment are reported elsewhere.19 For this study, scripted confederates performed the roles of anesthetist, scrub nurse, and circulating nurse.

The mixed-reality simulation environment included a synthetic patient model, a decommissioned C-arm and a trocar which were equipped with tracking markers to determine their spatial relationships via an optical tracking system. A 3D printed synthetic spine model derived from a real patient CT dataset was inserted into the mannequin, covered with gel wax and synthetic skin which allowed for realistic incision and suture. Using the patient's CT data and the spatial relationship between the C-arm, the patient model and the trocar, radiation-free simulated fluoroscopic x-ray images identical of the surgical scene are generated.20 An injection system was employed to ensure precise measurement of bone cement injection. Lastly, standard tools for VP were available, that is, scalpel, trocar, clamps, hammer, and suturing material.

Back to Top | Article Outline

Study Procedure

Participating surgeons were acquainted with the OR-set up, the team and the patient case. They subsequently performed a VP procedure under experimental conditions. Assessments started with the team time out and ended after suturing and surgeon stepping back from the table. Afterwards, each participant filled out a questionnaire. All confederate actors had a medical background and adhered to the predefined script. The procedure was captured on audio- and video recordings.

Back to Top | Article Outline

Measures

Surgeons’ NTS Assessment

Two trained observers rated surgeons’ intraoperative behaviors with the Observational Teamwork Assessment for Surgery (OTAS) scale. OTAS is a well-established and valid tool for OR teamwork assessment.21 It assesses NTS on five dimensions (communication, coordination, cooperation/backup behavior, leadership, and monitoring/situational awareness) on a seven-point Likert scale (from “0 = problematic behavior” to “6 = exemplary behavior”). OTAS ratings of surgeons’ NTS were established by two trained observers based on video and audio recordings. Interobserver agreement was high with intraclass correlation coefficient = 0.83.

Back to Top | Article Outline

Surgeons’ TS Assessment and Surgical Outcomes

Surgical performance was determined with the Objective Structured Assessment of Technical Skill (OSATS). OSATS is valid and reliable22 and has been applied in spine surgery.23 OSATS consists of three parts: 10-item task-specific rating scale (range 0–10), six-item global rating scale (range 6–30), and pass–fail rating.

For surgical outcome assessment we developed a measure based on literature review and interviews with five senior spine surgeons.19 It consists of a 14-item rating scale on trocar positioning with items on No-Go-Areas harmed, number of positioning attempts, rating of trocar depth, and others. Overall sum scores range between 0 and 22 points with higher scores indicative of better outcome. The scale can be found as Appendix 1, http://links.lww.com/BRS/B445.

TS performances were evaluated by an orthopedic expert (author C.M.) who had >15 years of experience in VP and >1000 VPs performed. He was completely blinded toward participants′ identity. All TS ratings were based on video and audio recordings, a fully interactive 3D virtual reconstruction playback, a postoperative CT showing the cement injection, and the actual printed and treated bone model.

Back to Top | Article Outline

Surgeons′ Experience

Surgical experience was based on participants′ professional tenure (years since acquisition of license to practice) and number of VPs performed.

Back to Top | Article Outline

Study Sample

Eleven surgeons were included, with eight being male (72.7%). Six participants were trauma surgeons, three orthopedic surgeons, and two trauma and orthopedic specialists. Their working experience ranged between 0 years and 33 years (mean [M] = 7.82, standard deviation [SD] = 9.38). Numbers of VPs performed were between 0 and 200 (M = 35.0, SD = 62.75). For NTS, OTAS score ranged between 1.70 and 4.60 with a mean score of 3.01 (SD = 1.10). Regarding TS, task-specific OSATS score ranged between 5 and 10 with mean score of 8.09 (SD = 1.87) and mean global OSATS was 25.09 (SD = 3.88, range 20–30). Concerning the third OSATS outcome, 6 participants (54.5%) were evaluated with “pass,” whereas 5 (45.5%) failed. For surgical outcome, mean score of trocar positioning was 17.09 (SD = 4.91, range 8–22).

Back to Top | Article Outline

Data Analysis

Data were tested for normality using the Kolmogorov–Smirnov test. As most variables were not normally distributed, nonparametric tests were utilized for testing our study objectives. For determination of relationships, we applied one-tailed Kendall's tau b; for group difference tests, we used one-tailed Mann–Whitney U tests. Kendall′s tau b were converted to r2.24 Values of P < 0.05 were considered statistically significant. SPSS 25 (IBM Corp., Armonk, NY) was used.

Back to Top | Article Outline

RESULTS

We first tested for relationships between NTS and TS: OTAS scores correlated positively with task-specific OSATS scores (τ = 0.63, r2 = 0.70, p = 0.006) and trocar positioning scores (τ = 0.60, r2 = 0.65, P = 0.007). Therefore, surgeons with better nontechnical behaviors also achieved better technical performance and surgical outcomes (see Table 1). Association with global OSATS scores was close to significance (τ = 0.37, r2 = 0.30, P = 0.065). Regarding associations between technical performance and surgical outcome, we observed significant correlations between OSATS task-specific and trocar positioning scores (τ = 0.67, r2 = 0.75, P = 0.004) as well as global scores and trocar positioning scores (τ = 0.73, r2 = 0.83, P = 0.002). Therefore, surgeons with better performance achieved superior outcomes.

TABLE 1

TABLE 1

For group differences between OSATS’ “pass” versus “fail” evaluation, nontechnical performance scores were significantly higher in “pass” group (M = 3.73, SD = 0.88) compared to “fail” group (M = 2.14, SD = 0.58; U = 3.00, P = 0.013). Respective scores of all five OTAS dimensions were significantly higher for “pass” group (see Appendix 2, http://links.lww.com/BRS/B445 for details).

We then determined associations between surgeons′ experience and intraoperative performance measures. Professional tenure correlated significantly with nontechnical performance, that is, OTAS overall score (τ = 0.71, r2 = 0.81, P = 0.001), task-specific OSATS scores (τ = 0.49, r2 = 0.48, P = 0.022), global OSATS scores (τ = 0.47, r2 = 0.45, P = 0.024), and trocar positioning scores (τ = 0.55, r2 = 0.58, P = 0.011). Identically, number of VPs performed correlated significantly with OTAS scores (τ = 0.72, r2 = 0.82, P = 0.001), task-specific OSATS scores (τ = 0.50, r2 = 0.50, P = 0.022), global OSATS scores (τ = 0.52, r2 = 0.53, P = 0.016), and trocar positioning scores (τ = 0.52, r2 = 0.53, P = 0.016). All correlations are depicted in Table 1.

We finally ran partial correlations for NTS and TS while controlling for both surgical experience measures, respectively. After taking account of professional tenure, correlations of OTAS scores with task-specific OSATS scores were τ = 0.45 (r2 = 0.42, P = 0.094), τ = 0.05 (r2 = 0.01, P = 0.441) with global OSATS scores, and τ = 0.36 (r2 = 0.29, P = 0.152) with trocar positioning scores. Associations of OTAS scores controlled for number of VPs performed were τ = 0.44 (r2 = 0.41, P = 0.100) with taskspecific OSATS scores, τ = −0.01 (r2 = 0.00, P = 0.487) with global OSATS scores, and τ = 0.38 (r2 = 0.32, P = 0.137) with trocar positioning scores. See Appendix 3, http://links.lww.com/BRS/B445 for details.

Back to Top | Article Outline

DISCUSSION

We investigated the relationship between technical and NTS in a simulated spine procedure. Our results revealed positive correlations such that higher NTS levels were related with increased TS, and vice versa. We further determined the influence of surgical experience that attenuated the strength of the association. To the best of our knowledge, this study is the very first in the field of spine surgery that comprehensively determines perioperative technical and NTS as well as experience. It thus contributes in several ways to the limited literature base.

First, concerning the role of surgical experience, we investigated the effect of experience on the relationship between technical and NTS. Previous studies showed inconsistent results: a simulation study including surgeons with 2 to 34 years of practice, surgical experience was found to have an effect on NTS and TS.25 In the same line, in vascular surgeons an investigation reported significant correlations between technical and NTS with significant differences between different experience groups.11 A third study reported a lack of respective correlations for urological surgeons.13 Further studies reported moderate to strong correlations between TS and NTS in simulation studies with anesthesiology residents,15 residents performing an open gastrojejunostomy,16 and novice surgeons conducting an ureteroscopy.26 Another study showed a non-linear relationship between NTS and experience, with NTS peaking around the time of fellowship in a sample of various surgical specialties and different experience levels.12 Summing up, it is possible that these inconsistencies can be attributed to the influence of experience on the relationship between NTS and TS. The results of our study resonate well with this assumption such that experience was highly related to NTS as well as TS.11,27 Despite the observation that NTS and TS improve through surgical practice, other post-hoc explanations may apply. Older, more experienced surgeons have a higher likelihood of previously completed NTS trainings, given the observation that NTS do not improve over time without practice and training.28 A further explanation stems from Cognitive load theory29: experienced surgeons have more mental resources to focus on NTS since their working memory is less occupied by technical performance demands. Moreover, intraoperative stress is proposed to contribute to poor TS.3 Hence, the relationship between NTS and TS is influenced by external stressors and individual coping behaviors.25,30

Owing to its pilot character, our study has some limitations. First, as this was a simulation study it is possible that participants did not perform to their full potential. Yet, the simulation procedure and environment were based on a rigorous, in-depth development process to reproduce a realistic OR setting with authentic OR team members.19 A systematic review on skills transfer after surgical simulation-based training indicates good transferability to real life surgery.31 We used ad-hoc teams instead of fixed teams that know each other; this may have influenced the results, as fixed teams show better NTS than ad-hoc teams.10 For TS, few surgeons attained maximum scores with ceiling effects. However, it is inevitable that experienced surgeons will achieve maximum performance levels in a routine procedure with moderate demands. In addition, our sample size is limited. As we used a convenience sampling approach a self-selection of the participants cannot be excluded. Regarding our assessment instruments, further research on the validity and reliability of the newly developed scale for surgical outcome is required. Additionally, we did not assess stress as contributing factor. This study was a cross-sectional study. Therefore, we cannot draw any inferences on the causality of the scores pertaining to surgeons′ NTS, TS, and experience. Nonetheless, our study has various strengths such as its comprehensive assessment of technical and NTS as well its employment of a full-scale OR team work environment.

The findings of this study advance future comprehensive assessment of TS and NTS in spine surgery. Future studies should investigate the effect of surgical experience on the relationship between NTS and TS with bigger sample sizes and across various surgical settings. Although simulation studies are a good opportunity to assess skills without harming patients, future studies may include real OR performance and patient outcomes for comparison. Moreover, our study simulated a VP, which is minimally invasive. Future studies may also include open surgeries as they show differences to minimally invasive surgeries regarding NTS.10 Finally, future investigations should include an assessment of intraoperative stress and learning curves when investigating NTS.

We examined the relationship between NTS and TS as well as surgical experience using a newly developed mixed-reality simulation environment for VP. We applied standardized observational tools and were able to demonstrate moderate to strong relationships specifically to spine surgery. Our results stress the importance of controlling the effect of professional experience when investigating NTS and TS. Moreover, our work provides further insights in establishing assessment and training environments to improve safety and patient care in spine surgery.

Back to Top | Article Outline

Acknowledgments

The authors acknowledge Angelika Richter und Nicole Ostermeier for supporting us as confederates. Moreover, the authors like to thank Michael Wengler, Aljoscha Kullmann, and Alexander Winkler for supporting us in the scenario setup.

Back to Top | Article Outline

References

1. Flin R, Mitchell L. Safer Surgery: Analysing Behaviour in the Operating Theatre. Boca Raton, FL: CRC Press; 2017.
2. Siu J, Maran N, Paterson-Brown S. Observation of behavioural markers of non-technical skills in the operating room and their relationship to intra-operative incidents. Surgeon 2016; 14:119–128.
3. Hull L, Arora S, Aggarwal R, et al. The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 2012; 214:214–230.
4. Yule S, Flin R, Paterson-Brown S, et al. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 2006; 139:140–149.
5. Panesar SS, Carson-Stevens A, Mann BS, et al. Mortality as an indicator of patient safety in orthopaedics: lessons from qualitative analysis of a database of medical errors. BMC Musculoskelet Disord 2012; 13:93.
6. Bible JE, Shau DN, Kay HF, et al. Are low patient satisfaction scores always due to the provider?: Determinants of patient satisfaction scores during spine clinic visits. Spine (Phila Pa 1976) 2018; 43:58–64.
7. Trumble WR, Stevenson A, Brown L. Shorter Oxford English dictionary on historical principles. 5th edOxford; New York: Oxford University Press; 2002.
8. McCulloch P, Mishra A, Handa A, et al. The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Qual Saf Health Care 2009; 18:109–115.
9. Mishra A, Catchpole K, Dale T, et al. The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surg Endosc 2008; 22:68–73.
10. Gjeraa K, Spanager L, Konge L, et al. Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc 2016; 30:5185–5199.
11. Black SA, Nestel DF, Kneebone RL, et al. Assessment of surgical competence at carotid endarterectomy under local anaesthesia in a simulated operating theatre. Br J Surg 2010; 97:511–516.
12. Gostlow H, Marlow N, Thomas MJW, et al. Non-technical skills of surgical trainees and experienced surgeons: Non-technical skills of surgical trainees and experienced surgeons. Br J Surg 2017; 104:777–785.
13. Lee JY, Mucksavage P, Canales C, et al. High fidelity simulation based team training in urology: a preliminary interdisciplinary study of technical and nontechnical skills in laparoscopic complications management. J Urol 2012; 187:1385–1391.
14. Coelho G, Defino HLA. The role of mixed reality simulation for surgical training in spine: phase 1 validation. Spine (Phila Pa 1976) 2018; 43:1609–1616.
15. Riem N, Boet S, Bould MD, et al. Do technical skills correlate with non-technical skills in crisis resource management: a simulation study. Br J Anaesth 2012; 109:723–728.
16. Rao R, Dumon KR, Neylan CJ, et al. Can simulated team tasks be used to improve nontechnical skills in the operating room? J Surg Educ 2016; 73:e42–e47.
17. Brunckhorst O, Shahid S, Aydin A, et al. Simulation-based ureteroscopy skills training curriculum with integration of technical and non-technical skills: a randomised controlled trial. Surg Endosc 2015; 29:2728–2735.
18. Cheng A, Kessler D, Mackinnon R, et al. Reporting guidelines for health care simulation research: extensions to the CONSORT and STROBE statements. Adv Simul 2016; 1:25.
19. Pfandler M, Stefan P, Wucherer P, et al. Stepwise development of a simulation environment for operating room teams: the example of vertebroplasty. Adv Simul 2018. 3.
20. Stefan P, Habert S, Winkler A, et al. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg 2018; 13:1335–1344.
21. Hull L, Arora S, Kassab E, et al. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 2011; 212:234–243. e5.
22. Hatala R, Cook DA, Brydges R, et al. Constructing a validity argument for the Objective Structured Assessment of Technical Skills (OSATS): a systematic review of validity evidence. Adv Health Sci Educ 2015; 20:1149–1175.
23. Boody BS, Rosenthal BD, Jenkins TJ, et al. The effectiveness of bioskills training for simulated open lumbar laminectomy. Glob Spine J 2017; 7:794–800.
24. Walker DA. JMASM9: converting Kendall's Tau for correlational or meta-analytic analyses. J Mod Appl Stat Methods 2003; 2:525–530.
25. Wetzel CM, Black SA, Hanna GB, et al. The effects of stress and coping on surgical performance during simulations. Ann Surg 2010; 251:171–176.
26. Brunckhorst O, Shahid S, Aydin A, et al. The relationship between technical and nontechnical skills within a simulation-based ureteroscopy training environment. J Surg Educ 2015; 72:1039–1044.
27. Anderson DD, Long S, Thomas GW, et al. Objective Structured Assessments of Technical Skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res 2016; 474:874–881.
28. Dedy NJ, Zevin B, Bonrath EM, et al. Non-technical skills of surgery residents: does experiential learning lead to competence? J Am Coll Surg 2013; 217:S116.
29. Van Merrienboer JJG, Sweller J. Cognitive load theory in health professional education: design principles and strategies: Cognitive load theory. Med Educ 2010; 44:85–93.
30. Krage R, Zwaan L, Tjon Soei Len L, et al. Relationship between non-technical skills and technical performance during cardiopulmonary resuscitation: does stress have an influence? Emerg Med J 2017; 34:728–733.
31. Dawe SR, Pena GN, Windsor JA, et al. Systematic review of skills transfer after surgical simulation-based training. Br J Surg 2014; 101:1063–1076.
Keywords:

backup behavior; communication; cooperation; coordination; experience; leadership; mixed-reality; monitoring; non-technical skills; simulation; situational awareness; technical skills; vertebroplasty

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.