Outcomes of catheter-related infections were reported by four studies (see Supplemental Digital Table 4 at http://links.lww.com/ACADMED/A56).11,36,37,40 However, numerator and denominator for events were missing in one study, and that study was excluded from analysis.37 Simulation training was not associated with significant differences in catheter-related infection risk (Figure 3, panel 3D).
MERSQI scores for randomized studies (13; SD = 2) were not significantly different from scores for nonrandomized studies (14.9 ± 0.22; P = .16). Statistical testing using the Begg test found no evidence for funnel plot asymmetry in any of the patient outcomes (P > .05).
Our systematic review and meta-analysis identified a number of publications supporting the value of simulation-based education for CVC. Benefits were seen in learner performance outcomes on simulators, learner knowledge, and confidence. Benefits were also seen in clinical outcomes such as decrease in the number of needle passes and decrease in the risk of pneumothoraces. These clinical benefits were demonstrated by studies of nonrandomized, two-group designs but were less convincing in studies that were randomized controlled trials. Last, regarding the risk of catheter-related infections, we found no statistically significant differences in study participants who received simulator training compared with those in control groups who did not.
To our knowledge, no previous systematic review on the effect of simulation-based education for CVC has been undertaken. A previous systematic review on surgical simulation concluded that benchtop simulation may be better than no training or standard training, but only four studies were included for that portion of the review.43 Our findings extend the work of others by providing current evidence pertaining specifically to the insertion of central venous catheters. Achieving competency in this procedure is a stated objective for a number of postgraduate training programs.44–47
The methodologies of the studies in the body of literature on which this meta-analysis is based have a number of limitations. First, the majority of the studies were observational. Pooled results are therefore subject to the effect of both measured and unmeasured confounders. Nonetheless, within the confines of the study design limitations, the mean MERSQI score for assessment of the quality of the studies included in this meta-analysis was quite high (12.6; SD = 2.3). To place this score in context, one may compare this score with a mean MERSQI score of 10.7 reported for a sample of manuscripts on medical education accepted for publication by one general medical journal.48
Second, statistical testing of funnel plots for asymmetry was significant in all three pooled learner outcomes, raising the possibility of publication bias. Although no significant asymmetry was detected for clinical outcomes, absence of significant asymmetry does not mean that publication bias was absent.49
Third, limited data are available on skill retention and direct skill transfer for CVC. Data from the surgical literature indicate that skills acquired by simulation-based education are transferable to the operative setting.50 Whether or not results are generalizable to a bedside procedure such as CVC is unknown.
Fourth, we were not able to isolate the effects of the technical aspects of simulation-based education from those of didactic teaching and other infection control cointerventions. Complications from central venous catheters arise beyond the act of insertion itself. Other aspects of CVC such as availability of proper equipment and supplies, catheter maintenance, and monitoring often require coordinated interprofessional team effort.51 Therefore, the extent of benefits gained from a simulation-based educational intervention needs to be placed within the context of other hospital-wide educational efforts aimed at decreasing complications.
In addition, our review has limitations. First, despite work by others clearly demonstrating the effectiveness of providing feedback, repetitive practice, and providing learners with a range of task difficulty,55 we did not explore the effects of specific elements of curriculum design on outcomes. Our review suggests that whereas some of these elements were clearly included in the educational intervention (up to 45% of studies reported giving feedback), other features were rarely reported, such as presenting learners with a range of task difficulty. Second, some studies could not be included in the pooled analyses because of missing data, despite contacting the authors for more information. Third, unexplained heterogeneity was noted among several outcomes. Last, pooling effect sizes across study designs is problematic.17 We have therefore provided results for our meta-analyses, stratified by study design.
These limitations notwithstanding, this meta-analysis provides a detailed assessment of the effects of simulation-based education for CVC on learner and patient outcomes. Benefits seen in learner outcomes were consistent, with moderate effect sizes. In light of increasing concern regarding patient safety issues,56 we recommend that the implementation of simulation-based educational programs become a strategic priority for educating learners who insert central venous catheters. Assessing for the effectiveness of elements of instructional designs relating to learners, instructors, simulators, and environment was beyond the scope of our review. For future studies, we recommend that the academic research community focus its attention on the following questions:
* What is the effect of specific elements of instructional design such as mastery learning,25,40–42 simulator fidelity, and teaching environment on competency?
* What is the effect of simulation-based education on skill retention and skill transfer?
In conclusion, results from our meta-analysis support the use of simulation in the education of central venous catheter insertion. Simulation-based teaching for CVC should be made available for learners striving to attain competency in this procedure.
This study was funded in part by the Departments of Medicine and Surgery Research Development Fund from the University of Calgary. Paul Ronksley received funding from the Frederick Banting and Charles Best Canada Graduate Scholarship from the Canadian Institutes of Health Research. Dr. Ghali received funding from the Government of Canada Research Chair in Health Services Research and Senior Health Scholar award from Alberta Innovates-Health Solutions. The funding agencies had no role in the design and conduct of this study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, and approval of the manuscript.
1 Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282:861–866.
2 Okuda Y, Bond W, Bonfante G, et al. National growth in simulation training within emergency medicine residency programs, 2003–2008. Acad Emerg Med. 2008;15:1113–1116.
3 Ogden PE, Cobbs LS, Howell MR, Sibbitt SJ, DiPette DJ. Clinical simulation: Importance to the internal medicine educational mission. Am J Med. 2007;120:820–824.
4 Weinberg ER, Auerbach MA, Shah NB. The use of simulation for pediatric training and assessment. Curr Opin Pediatr. 2009;21:282–287.
5 Kneebone R. Simulation in surgical training: Educational issues and practical implications. Med Educ. 2003;37:267–277.
6 Matveevskii AS, Gravenstein N. Role of simulators, educational programs, and nontechnical skills in anesthesia resident selection, education, and competency assessment. J Crit Care. 2008;23:167–172.
7 Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein catheterization. Failure and complication rates by three percutaneous approaches. Arch Intern Med. 1986;146:259–261.
9 McGee DC, Gould MK. Preventing complications of central venous catheterization. N Engl J Med. 2003;348:1123–1133.
10 Kurrek MM, Devitt JH. The cost for construction and operation of a simulation centre. Can J Anaesth. 1997;44:1191–1195.
11 Sherertz RJ, Ely EW, Westbrook DM, et al. Education of physicians-in-training can decrease the risk for vascular catheter infection. Ann Intern Med. 2000;132:641–648.
12 Cohen ER, Feinglass J, Barsuk JH, et al. Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc. 2010;5:98–102.
13 Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med. 2009;151:264–269.
14 Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–1009.
15 Hedges L, Olkin I, eds. Statistical Methods for Meta-Analysis. San Diego, Calif: Academic Press; 1985.
16 Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to Meta-Analysis. Chichester, West Sussex, UK: John Wiley & Sons, Ltd; 2009.
17 Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, West Sussex, UK: John Wiley & Sons, Ltd; 2008.
18 Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York, NY: Academic Press; 1988.
19 Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–560.
20 DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–188.
21 Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–1101.
22 Guyatt G, Rennie D. User's Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, Ill: American Medical Association Press; 2002.
24 Liachopoulou AP, Synodinou-Kamilou EE, Deligiannidi PG, Giannakopoulou M, Birbas KN. Nurses' training and confidence on deep venous catheterization. Technol Health Care. 2008;16:159–169.
25 Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70–76.
26 Carvalho P. Early introduction to central line placement: A curriculum for medical students. Med Educ. 2007;41:1098–1099.
27 Britt RC, Reed SF, Britt LD. Central line simulation: A new training algorithm. Am Surg. 2007;73:680–682.
28 Lee AC, Thompson C, Frank J, et al. Effectiveness of a novel training program for emergency medicine residents in ultrasound-guided insertion of central venous catheters. CJEM. 2009;11:343–348.
29 Millington SJ, Wong RY, Kassen BO, Roberts JM, Ma IW. Improving internal medicine residents' performance, knowledge, and confidence in central venous catheterization using simulators. J Hosp Med. 2009;4:410–416.
30 Sanchez LD, Delapena J, Kelly SP, Ban K, Pini R, Perna AM. Procedure lab used to improve confidence in the performance of rarely performed procedures. Eur J Emerg Med. 2006;13:29–31.
32 Macnab AJ, Macnab M. Teaching pediatric procedures: The Vancouver model for instructing Seldinger's technique of central venous access via the femoral vein. Pediatrics. 1999;103:E8.
33 Ault MJ, Rosen BT, Ault B. The use of tissue models for vascular access training. Phase I of the procedural patient safety initiative. J Gen Intern Med. 2006;21:514–517.
34 Rosen BT, Uddin PQ, Harrington AR, Ault BW, Ault MJ. Does personalized vascular access training on a nonhuman tissue model allow for learning and retention of central line placement skills? Phase II of the procedural patient safety initiative (PPSI-II). J Hosp Med. 2009;4:423–429.
35 Velmahos GC, Toutouzas KG, Sillin LF, et al. Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. Am J Surg. 2004;187:114–119.
36 Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central line simulation before the ICU experience. Am J Surg. 2009;197:533–536.
37 Miranda JA, Trick WE, Evans AT, Charles-Damte M, Reilly BM, Clarke P. Firm-based trial to improve central venous catheter insertion practices. J Hosp Med. 2007;2:135–142.
38 Lenhard A, Moallem M, Marrie RA, Becker J, Garland A. An intervention to improve procedure education for internal medicine residents. J Gen Intern Med. 2008;23:288–293.
39 Martin M, Scalabrini B, Rioux A, Xhignesse MA. Training fourth-year medical students in critical invasive skills improves subsequent patient safety. Am Surg. 2003;69:437–440.
40 Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169:1420–1423.
41 Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397–403.
42 Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697–2701.
43 Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation: A systematic review. Ann Surg. 2006;243:291–300.
48 Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for medical education research study quality instrument scores: Quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med. 2008;23:903–907.
49 Ioannidis JP, Trikalinos TA. The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey. CMAJ. 2007;176:1091–1096.
50 Sturm LP, Windsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ. A systematic review of skills transfer after surgical simulation training. Ann Surg. 2008;248:166–179.
51 Berenholtz SM, Pronovost PJ, Lipsett PA, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32:2014–2020.
52 Hind D, Calvert N, McWilliams R, et al. Ultrasonic locating devices for central venous cannulation: Meta-analysis. BMJ. 2003;327:361.
53 Feller-Kopman D. Ultrasound-guided central venous catheter placement: The new standard of care? Crit Care Med. 2005;33:1875–1877.
55 Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach. 2005;27:10–28.