Secondary Logo

Journal Logo

Education, simulation and resuscitation

Simulation-based medical education training improves short and long-term competency in, and knowledge of central venous catheter insertion

A before and after intervention study

Cartier, Vanessa; Inan, Cigdem; Zingg, Walter; Delhumeau, Cecile; Walder, Bernard; Savoldelli, Georges L.

Author Information
European Journal of Anaesthesiology: August 2016 - Volume 33 - Issue 8 - p 568-574
doi: 10.1097/EJA.0000000000000423

Abstract

Introduction

Adverse events related to central venous catheters (CVCs) are well known and education and training has been shown to be effective in their prevention.1,2 One major complication after CVC insertion is a central line-associated bloodstream infection (CLABSI). The reported incidence density of CLABSI varies between 1.5 and 6.5 infections per 1000 CVC-days in the United States.3 Approximately, 1800 CVCs are used at the University Hospitals of Geneva (HUG), a University-affiliated primary and tertiary care hospital with 950 acute care beds (36 in intensive care) each year; 90% are inserted either by an anaesthesiologist or an ICU physician (45% each).

The prevalence of bloodstream infection is 1.1% (0.9–1.4%) as measured in repeated surveys.4,5 In 2006, CLABSI rates were 3.8 CLABSI/1000 CVC-days hospital wide.6 The incidence densities for CVCs placed by anaesthesiologists and intensive care physicians were 4.9 and 2.7 CLABSI/1000 CVC-days, respectively.6 Comprehensive prevention strategies addressing practice from insertion to removal of CVCs have been shown to be effective in CLABSI prevention.7,8 Thus, education and training of all levels of healthcare professionals involved in CVC insertion and care has been recommended.9–11 Recent studies have shown that mechanical complications and CLABSI were reduced after simulation-based CVC-insertion training.1,12

A multimodal quality improvement project with the acronym REDCO-CVC (REDuction of COmplication related to CVC) on CVC insertion and care was established in the HUG in 2007 and implemented from 2008.7 The aim of this study was to evaluate short and long-term competency in, and knowledge of residents in CVC insertion before and after simulation-based medical education (SBME) training.

Material and methods

Ethical approval for this study (Ethical committee No. NAC 07-023) was provided by the Ethical Committee NAC University Hospitals of Geneva and Geneva Faculty of Medicine, University of Geneva, Geneva, Switzerland (Chairman Professor J.S. Lacroix) on 4 September 2007. The study participants provided written consent.

Study participants

Residents from the Department of Anaesthesiology, University Hospitals of Geneva, were eligible on a voluntary basis for training, if they had already performed at least one CVC insertion using the Seldinger technique on a patient.

Study design

From May 2008 to December 2008, residents were trained in small groups of four to six participants. Competency of CVC insertion was assessed by video recording. Competency was re-assessed by video recording after a period of more than 2 years (sustainability phase). Within 3 days before (pretraining) and within 3 days after (posttraining) training, knowledge was tested by a standardised multiple-choice questionnaire (MCQ) with identical questions pretraining and posttraining for all participants.

Competency assessment and knowledge testing

Competency assessment was performed by video recording of individual CVC insertions on a mannequin (Laerdal IV Torso). For this purpose, the participants were asked to perform CVC insertion by using their familiar practice. The educational and training programme took place in the simulation laboratory of HUG (SimulHUG, http://simulationmedicale.hug-ge.ch). The laboratory has a debriefing/theory room and a virtual operating room, which is equipped with audio recording, and video cameras showing views from different angles.

The 22 questions of the MCQ addressed knowledge of indications, insertion technique and potential complications of CVC use. The MCQ was developed based on the available literature of best practice at the time. The content was validated by four senior anaesthesiologists by a modified Delphi technique before it was trialled with four anaesthesiologists to ensure that the questions were comprehensible and in line with the content of the education and training programme.

Education and training of central venous catheter insertion competency

Competency training focused on technical skills, aseptic technique and work organization. Knowledge acquisition focused on the prevention of acute mechanical and infectious complications. The emphasis was on implementing appropriate hand hygiene; thorough skin preparation with coloured alcohol-based chlorhexidine gluconate; use of maximal sterile barrier precautions with masks, caps, sterile gloves, sterile gowns and large drapes; working by the concept of minimal handling and the use of a new single-use kit for CVC insertion (BBraun Company, Switzerland). This kit, which was designed by the REDCO-CVC team, has two distinct layers: an upper layer with the material for skin preparation and a lower layer with the necessary equipment for CVC insertion.10 To minimise the risk of chlorhexidine contaminating equipment, the skin preparation is completed and the upper layer of the kit is removed before accessing the lower layer containing the CVC insertion equipment. Three teaching formats were combined: formal lecture, hands-on training on the mannequin and tests (knowledge test and competency assessment).

The formal lecture was based on Centre for Disease Control and Prevention guidelines.10 The training consisted of an interactive slide presentation and case-based group discussions. After watching examples from the pretraining competency assessment, a best practice video of CVC insertion in a patient was shown and discussed among the participants. Attention was given to correct patient positioning, CVC equipment preparation and patient monitoring; timing and technique of hand hygiene; surgical skin antisepsis; application of maximal sterile barrier precautions; minimal handling and prevention of acute mechanical complications (arterial puncture, pneumothorax, incorrect CVC position). Catheter care and removal were also addressed. The participants were given a detailed checklist that included every single step of CVC insertion, from patient preparation to the application of the dressing.

Competency training was performed in the simulation laboratory. Every participant's skills were recorded and later reviewed in the debriefing room. The participants took turn as operators, assistants or observers by using the checklist. Direct feedback by the instructors consisted of corrective constructive feedback and positive reinforcement. The assessments were adherence to the sequence as described by the checklist, the correct use of the new single-use kit for CVC insertion, the hand hygiene timing during the insertion procedure, correct skin antisepsis by applying coloured alcohol-based chlorhexidine gluconate, the use of maximal sterile barrier precautions, correct patient positioning for CVC insertion and the presence of an assistant wearing a surgical mask and a cap.

Several cognitive aids such as an intranet website and a detailed procedural checklist were available in clinical practice between the initial training and the testing in the sustainability phase. Since the completion of this study a freely accessible website with detailed video presentations of all aspects of CVC insertion has been created - www.carepractice.net.

Measurements of central venous catheter insertion competency

Several scores were used to assess skills using the video recordings: a global rating scale (GRS) of technical skills (Appendix 1: Technical skills score, http://links.lww.com/EJA/A87) adapted from a validated rating form used to assess surgical skills (minimum score 8, maximum score 40)13; a checklist assessing hand hygiene timing [Appendix 2: Hand Hygiene compliance score (minimum score 0, maximum score 8), http://links.lww.com/EJA/A87] and a checklist focusing on maximal sterile barrier precautions and minimal handling [Appendix 3: Checklist compliance score (minimum score 0; maximum score 20), http://links.lww.com/EJA/A87]. Two raters (who were trained by a senior anaesthesiologist) reviewed the videos and scored the participants’ performances. Inter-rater reliability was assessed using 22 randomly selected videotapes. For the pretraining and posttraining competency assessments, all videos were randomly numbered. Reviewers were unaware of the participant's identity or the recording sequence. Competency measurements in the sustainability phase were performed by one reviewer who was blinded to the participant's identity.

Outcomes

The primary study outcome was competency as measured by the global rating scale of technical skills score, the hand hygiene compliance score and the checklist compliance score. Secondary outcome was knowledge as measured by the pretraining and posttraining MCQ.

Sample size

An estimation of the minimum sample size was computed a priori based on previous experiences in the field of psychology and education. In such areas, an effect size of above 1.0 SD is considered large and significant for an educational intervention.14 Hence, assuming such an effect size of 1.0 SD, samples of above 15 participants are usually sufficient to detect a difference for a paired sample with a minimum power of 0.8 and P less than 0.05 (two tailed). As the educational intervention was integrated in residency training, we decided to include all eligible residents during the recruitment period. This would allow us to guarantee a sufficient sample size both for the immediate posttraining phase and the sustainability phase allowing for a degree of attrition.

Statistics

Distribution of the outcome variables was analysed using the Shapiro–Wilk W test for normality. Statistical comparisons were performed using matched paired Student's t test or Wilcoxon signed-rank test as appropriate. Inter-rater reliability was assessed using κ or Spearman's ρ coefficient. When a specific aspect of the procedure could not be well identified in the video recordings, the reviewer could score that specific part as ‘not done’ or ‘not observable’; however, the two scores were modelled as zero in the analysis.

All statistical analyses were performed using STATA Release 13.0 (Stata Statistical Software: Release 13.0; Stata Corporation, College Station, Texas, USA).

Results

Between May and December 2008, 55 residents from the Department of Anaesthesia were invited to participate in the study (Fig. 1). Of these, 17 were not eligible [16 with limited contracts, one with an exclusion criterion (no experience)] and one refused to participate. The remaining 37 participants completed pretraining and posttraining tests. Nineteen participants were lost to follow-up because they had left the hospital before starting the tests in the sustainability phase, leaving 18 participants who completed all three parts: before training, after training, and in the sustainability phase. Participants’ ages averaged 30 years and most had performed more than five CVC insertions before training (Table 1).

Fig. 1
Fig. 1:
Flow chart.
Table 1
Table 1:
Participants’ characteristics at the time of recruitment

A total of 96 videos were reviewed and scored. The inter-rater reliability, based on 22 randomly selected recordings from pretraining and posttraining, was 0.59 for the GRS-technical skills score and 0.75 for the checklist compliance score (Spearman's ρ). For hand hygiene compliance, the κ score was 0.60 pretraining (P = 0.003) and 0.80 posttraining (P = 0.002). There were no missing data from the GRS-technical skills score, the hand hygiene compliance score or the checklist compliance score.

The average global rating scale of technical skills scores (Fig. 2), hand hygiene compliance scores (Fig. 3) and checklist compliance scores assessing maximal sterile barrier precautions and minimal handling (Fig. 4) all significantly improved after training, and there was evidence that some of the improvements were maintained in the sustainability phase.

Fig. 2
Fig. 2:
Thick line: median value; upper and lower edges of the box: upper and lower quartiles; whiskers: defined within SPSS as the most distant point which does not meet the definition of an outlying or extreme value. If there are no outliers or extreme values the whiskers represent the range. O: outlying points, defined as cases with values between 1.5 and 3 box lengths from the upper or lower edge of the box. *Extreme points, defined as cases with values more than three box lengths from the upper or lower edge of the box. There were no extreme values. n, number; SD, Standard Deviation.
Fig. 3
Fig. 3:
Thick line: median value; upper and lower edges of the box: upper and lower quartiles; whiskers: defined within SPSS as the most distant point which does not meet the definition of an outlying or extreme value. If there are no outliers or extreme values the whiskers represent the range. O: outlying points, defined as cases with values between 1.5 and 3 box lengths from the upper or lower edge of the box. *Extreme points, defined as cases with values more than three box lengths from the upper or lower edge of the box. There were no extreme values. n, number; SD, Standard Deviation.
Fig. 4
Fig. 4:
Thick line: median value; upper and lower edges of the box: upper and lower quartiles; whiskers: defined within SPSS as the most distant point which does not meet the definition of an outlying or extreme value. If there are no outliers or extreme values the whiskers represent the range. O: outlying points, defined as cases with values between 1.5 and 3 box lengths from the upper or lower edge of the box. Extreme points, defined as cases with values more than three box lengths from the upper or lower edge of the box. There were no extreme values. n, number; SD, Standard Deviation.

Seventy-four MCQs were completed and could be analysed. The percentage of correct answers improved from 76.0% [±(SD) 7.9] before training to 87.7% (±4.4) shortly after training (P < 0.001).

Discussion

Our study has demonstrated that technical skills, adherence to hand hygiene, maximal sterile barrier precautions and minimal handling significantly improved after a multimodal educational intervention based on SBME training. CVC competency in the sustainability phase, assessed on average 34 months after training, decreased but was still significantly higher when compared with the pretraining phase.

Compliance with hand hygiene was the most obvious improvement after training, perhaps because participants readily recognised its importance and rapidly adopted the recommended hand hygiene technique. The global technical skills, compliance scores and knowledge all considerably improved in the short-term after SBME training. This study is the first to evaluate and confirm long-term skills retention in relation to CVC insertion after SBME based training. Barsuk et al.15 assessed knowledge and skills in CVC use in 80% of their participants after 6 months, but only 32% after 12 months. Their participants demonstrated good retention scores (more than 82% after 6 months and 87% after 12 months). In our study, we observed a considerable decrease of CVC insertion competency after more than 2 years, but still higher than before training. This loss of competency occurred despite the regular exposure to CVC insertion including bedside teaching. This observation suggests that residents need regular refresher courses and SBME training.

Our study has limitations. First, this investigation was a before and after study and not a parallel randomised controlled trial of two groups. Therefore, we cannot exclude that a potential control group would improve over time without SBME and that this control group would have a similar result in the sustainability phase. This potential Hawthorne effect (a behavioural change related to observation) in an environment with a safety policy cannot be excluded. Second, we focused on residents in a single centre and all participants were already aware of CVC Seldinger insertion techniques. Hence, the results may not be generalized to other centres, or other healthcare providers such as totally naïve or more experienced operators. Third, half of the participants were lost in the long-term follow-up and this may have introduced some selection bias. However, characteristics were similar for the participants who completed the whole study compared with the participants who did not completed the sustainability phase. Fourth, most scores in this study are not validated except the global rating scale of technical skills: in the setting of CVC insertion, no validated scores were available before starting this study. It is unclear how levels of statistical significance for these scoring methods relate to clinical practice. Nevertheless, all items of the scores are relevant for proper CVC insertion and prevention of complications. Fifth, only one rater reviewed and scored the video recordings during the sustainability phase but inter-rater reliability in the first phase was judged to be good enough to use only one rater in this stage. Finally, no reliable data were available on the number of bedside CVC insertions after training and before the assessment in the sustainability phase.

Despite these limitations, this study confirms the value of multimodal educational prevention strategies including SBME training. Our results are in line with the recent reports about the benefit of simulation-based training.7,8 Strategies to maintain a high level of competency could include structured feedback during supervised clinical practice and regular hands-on workshops in the simulation laboratory. Further studies should aim to confirm and improve upon the observed long-term effect on competency in CVC insertion.

Acknowledgements relating to this article

Assistance with the study: the authors would like to thank the anaesthesiology residents of the Geneva University Hospitals (Geneva, Switzerland) for their participation in this study. They would also like to thank José-Manuel Garcia and Nadia Elia, MD, University Hospitals of Geneva and Geneva Faculty of Medicine, University of Geneva, Geneva, Switzerland, for their assistance with the study.

Financial support and sponsorship: this work was supported by the Anaesthesiology Division, University Hospital of Geneva, Geneva, Switzerland, who allocated the required resources. The Anaesthesiology Division has received 80 sterile industrial kits for CVC insertion and 80 sterile catheters for this specific study from BBraun Company, Switzerland.

Conflicts of interest: none.

Presentation: preliminary data for this study were presented as a poster presentation at the annual congress of the European Society of Intensive Care Medicine (ESICM), 9 to 13 October 2010, CCIB Barcelona, Spain.

Comment from the Editor: BW is an deputy editor in chief to the European Journal of Anaesthesiology.

References

1. Barsuk JH, Cohen ER, Feinglass J, et al. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009; 169:1420–1423.
2. McGaghie WC, Issenberg SB, Cohen ER, et al. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011; 86:706–711.
3. Edwards JR, Peterson KD, Mu Y, et al. National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009. Am J Infect Control 2009; 37:783–805.
4. Sax H, Pittet D. the Swiss-NOSO NetworkInterhospital differences in nosocomial infection rates: importance of case-mix adjustment. Arch Intern Med 2002; 162:2437–2442.
5. Zingg W, Huttner BD, Sax H, et al. Assessing the burden of healthcare-associated infections through prevalence studies: what is the best method? Infect Control Hosp Epidemiol 2014; 35:674–684.
6. Zingg W, Sax H, Inan C, Cartier V, et al. Hospital-wide surveillance of catheter-related bloodstream infection: from the expected to the unexpected. J Hosp Infect 2009; 73:41–46.
7. Zingg W, Cartier V, Inan C, et al. Hospital-wide multidisciplinary, multimodal intervention programme to reduce central venous catheter-associated bloodstream infection. PloS one 2014; 9:e93898.
8. Eggimann P, Harbarth S, Constantin MN, et al. Impact of a prevention strategy targeted at vascular-access care on incidence of infections acquired in intensive care. Lancet 2000; 355:1864–1868.
9. O’Grady NP, Alexander M, Dellinger EP, et al. Guidelines for the prevention of intravascular catheter-related infections. Infect Control Hosp Epidemiol 2002; 23:759–769.
10. O’Grady NP, Alexander M, Dellinger EP, et al. Guidelines for the prevention of intravascular catheter-related infections. Centers for Disease Control and Prevention. MMWR Recomm Rep 2002; 51 (RR-10):1–29.
11. O’Grady NP, Alexander M, Burns LA, et al. Guidelines for the prevention of intravascular catheter-related infections. Am J Infect Control 2011; 39:S1–S34.
12. Barsuk JH, McGaghie WC, Cohen ER, et al. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009; 37:2697–2701.
13. Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an innovative ‘bench station’ examination. Am J Surg 1997; 173:226–230.
14. Cohen J. Statistical power analysis for the behavioral sciences. 1977; New York: Academic press, 24–27.
15. Barsuk JH, Cohen ER, McGaghie WC, et al. Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Acad Med 2010; 85:S9–S12.
© 2016 European Society of Anaesthesiology