Over the past decade, the use of simulation-based technology in teaching technical skills has generated much enthusiasm1 and is increasingly used in a variety of disciplines and specialties within medical education.2–6 In the case of central venous catheterization (CVC), physicians who have placed 50 or more catheterizations are half as likely to cause mechanical complications as those who have placed fewer catheterizations,7 another verification of the observation that experience and deliberate practice play an important role in the optimization of technical skills.8 Thus, it is not surprising that in a commonly performed procedure such as CVC, with a complication rate of greater than 15%,9 simulation is an attractive educational tool. Simulation allows learners an opportunity for deliberate practice without jeopardizing patient safety.8 Yet despite its theoretic appeal, the implementation of a simulation-based procedural curriculum can be a costly endeavor, requiring availability of space, material resources, and faculty time.3,10,11 Despite emerging data suggesting the cost-effectiveness of simulation-based education,11,12 in the absence of compelling evidence favoring its use, widespread implementation of simulation-based medical education for the insertion of central venous catheters may not be justified. On the other hand, strong evidence in support of simulation-based medical education would provide the impetus to widely implement this technology for teaching CVC and other procedures.
Recent research examining the use of simulation-based education for CVC has been undertaken. To date, no systematic review has been conducted to summarize this body of literature. Therefore, we performed the systematic review and meta-analysis reported here to evaluate the available evidence on the use of simulation-based education for CVC on learner and patient outcomes.
Data sources and searches
The methods of this systematic review and meta-analysis were specified in advance and documented in a protocol available from us. This study is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses standards.13 We sought relevant articles between January 1950 and May 2010 using the electronic databases of MEDLINE, PubMed, Education Resource Information Center, Excerpta Medica, Cochrane Central Register of Controlled Trials, and the Cumulative Index of Nursing and Allied Health Literature. A search strategy was developed with the assistance of a research librarian using the following search terms: catheterization, central venous; catheterization; catheter$; jugular veins; subclavian veins; and femoral veins. These terms were searched as subject headings, medical subject headings, and text words where appropriate. We combined these using the Boolean operator “and” with education terms: education; learning; teaching; and teach$. No language restriction was placed on our search. To maximize the sensitivity of our search, we did not limit our search to terms related to simulation or study type. We conducted a hand search for references in all included articles and relevant review articles. Screening of search results was done independently by two of us (I.M., M.B.). Titles and abstracts were reviewed for eligibility. For abstracts that did not provide sufficient information to determine eligibility, full-length articles were retrieved. Disagreement on inclusion or exclusion of articles was resolved by consensus.
We included primary research articles if they described a simulation-based educational intervention on central venous catheter insertion taught to learners. No restrictions were placed on either the discipline or training level of learners. We defined simulation-based educational intervention as education involving one or more of the following modalities: partial-task trainers (commercially available, homemade trainers, or animal models), standardized patients, full body task trainers, high-fidelity mannequins, virtual reality, or computer software.3 We considered all study designs with a control group that did not receive an educational intervention involving simulation, including the use of historical control and prepost studies, whereby the subject served as his or her own control. We excluded articles that did not report a control group and studies that did not report results comparing a group that received simulation teaching with a control group. We excluded studies that involved peripherally placed venous access devices. Studies reporting an educational intervention that did not involve teaching technical skills of central venous catheter insertion were also excluded. For example, studies reporting only on educational interventions on nontechnical skills such as dressing changes and use of equipment bundles were excluded. See Figure 1 for a diagram of the study selection process.
Data extraction and quality assessment
We categorized outcomes of interest into two types: learner outcomes and patient outcomes. Learner outcomes of interest included performance measures on simulators and/or on patients, use of full-size sterile drapes, proper hand washing, use of ultrasound, knowledge, confidence, and satisfaction.
Patient outcomes considered were clinical complications of CVC, including number of needle passes, arterial puncture, pneumothorax, and catheter-related infection.
Independent data abstraction was performed by two of us (I.M., M.B.) using a standardized data form (see Supplemental Digital Form 1 at http://links.lww.com/ACADMED/A55. Agreement on selection of potentially relevant articles was reported using the kappa statistic. Disagreements were resolved by consensus. From each included article, we extracted the following information: study design, learner characteristics, sample size, components of interventional curriculum, source of control group, type of outcome assessed, and method of assessment. Attempts were made to contact the authors for missing data.
We assessed the methodological quality of the simulation-based CVC educational intervention by using the validated Medical Education Research Study Quality Instrument (MERSQI).14 The MERSQI is a 10-item tool for the evaluation of quality of medical education studies, examining domains in study design, sampling, data type, validity of assessments, data analysis, and outcomes. The total MERSQI score ranges between 5 and 18. Higher scores indicate higher quality. For studies with items that are not “not applicable,” the MERSQI scores were adjusted to a standard denominator of 18.14
Data synthesis and statistical analysis
The effect estimates for the simulation-based intervention were calculated by the change in learner or patient outcomes from baseline to end of follow-up and compared between groups. Dichotomous outcomes were pooled as risk ratios and continuous outcomes as standardized mean differences (SMDs), measured by calculating Cohen d.15,16 Pre- and postintervention means and standard deviations (SDs) for each outcome of interest were extracted from the intervention and control group. In some cases, SDs for mean changes were not directly reported. In these instances, SDs for mean changes were calculated using the P value or 95% confidence intervals (CIs) for within-group means, using formulas provided in the Cochrane Handbook for Systematic Reviews of Interventions.17 An SMD of 0.2 is considered small effect size; 0.5, moderate; and 0.8, large.18
Each outcome was initially pooled with a fixed-effects model. To assess heterogeneity across studies, we calculated the I2 statistics.19 The I2 values of 25%, 50%, and 75% are considered low, moderate, and high, respectively.19 In instances of moderate or high heterogeneity, the analysis was redone with the DerSimonian and Laird20 random-effects model to obtain a pooled effect size. Finally, we assessed for the possible presence of publication bias by inspecting for asymmetry in the funnel plot, which is a scatter plot of the magnitude of effect size against a measure of its precision. Formal testing for funnel plot asymmetry was performed using the Begg test.21 Comparisons of MERSQI scores between studies of differing study designs were assessed using Student t tests. All analyses were performed using Stata version 11.0 (Stata Corp, College Station, Texas), SAS version 9.1 (SAS Institute, Cary, North Carolina), and Comprehensive Meta-Analysis version 2 (Biostat, Englewood, New Jersey).
We identified 1,351 articles for title and abstract screening (Figure 1). After applying inclusion and exclusion criteria, we excluded 1,241 articles. Agreement between reviewers on the exclusion of 1,241 articles and inclusion of 110 remaining publications for detailed evaluation showed a kappa of 0.87 (95% CI 0.82–0.92). The remaining 110 full-text articles were reviewed, and 21 articles were considered for our review. Interrater agreement for this stage was also high (kappa = 0.94; 95% CI 0.86–1.0).22 One article was withdrawn because of duplication of published data.23
Characteristics of the 20 studies included in this systematic review are listed in Table 1. All but two of these studies were single-center studies.24,25 Nine out of the 20 studies (45%) were of single-group pretest and posttest design.26–34 Two studies (10%) were randomized controlled trials,35,36 whereas nine studies (25%) were of nonrandomized, two-group design.11,24,25,37–42
The primary learner outcomes of interest available for pooling included learner performance on simulators, knowledge, and confidence. The primary patient outcomes available for pooling included number of needle passes and risk of arterial puncture, pneumothorax, and catheter-related infections.
Learners at various levels of training and from various specialties were represented (Table 1). Median sample size was 79 (interquartile range 20–103). The majority of training models used commercial partial-task trainers (11; 55%). Two studies used unspecified mannequins,11,37 two used noncommercial/homemade simulators,24,32 and two used human cadavers.31,39 Animal models were used in three studies.30,33,34 One study used virtual reality in addition to a homemade simulator model.24
With the exception of one study,32 all articles explicitly stated that the educational intervention included a didactic component. Ten of the 20 studies (50%) specified that demonstration of technique was given.28,29,31,34–38,40,41 Opportunity for practice was specified by all but four studies.11,30,32,33 Feedback was specified as being given by nine studies (45%).24,25,29,31,34,39–42 Only one study (5%) clearly presented the learners with a range of task difficulty, whereby learners had the opportunity to practice on sponge material prior to inserting central venous catheters on partial-task trainers.26 Only nine studies (45%) stated that the use of ultrasound was taught.25,26,28,29,33,34,40–42 Seven of these studies directly evaluated their learners on the use of ultrasound while performing central venous catheterization.25,26,28,34,40–42
Quality assessment using MERSQI scores showed a mean score of 12.6 (SD = 2.3) out of a maximum possible total score of 18 (range 9.0–15.6; see Supplemental Digital Table 1 at http://links.lww.com/ACADMED/A56).
Learner performance outcomes on simulators and patients
Eight studies reported learner performance outcomes on simulators, and four studies reported learner performance outcomes on patients (see Supplemental Digital Table 2 at http://links.lww.com/ACADMED/A56). Six studies reported overall performance differences on simulators between pre- and post educational intervention.25,28,29,34,41,42 Low heterogeneity was observed when these studies were combined in a meta-analysis (I2 = 19.3%, P = .29). All six studies reported significant improvements post educational intervention, with a pooled fixed-effects SMD of 0.60 (95% CI 0.45–0.76); see Figure 2, panel 2A. The funnel plot appears asymmetric, raising the possibility of publication bias (Begg P = .02).
Only one study reported overall learner performance on patients (see Supplemental Digital Table 2 at http://links.lww.com/ACADMED/A56).35 One study demonstrated a significant increase in the use of full-size sterile drapes post educational intervention.11 One study reported significant increase in the use of ultrasound.42 No improvement in hand washing was demonstrated. No pooling of data was possible with these outcomes.
Skill retention and skill transfer
Only one study evaluated for and demonstrated retention of technical skills with repeat testing done at three to four weeks' time.34 One study directly correlated overall performance on simulators with performance on patients.35 That study reported no significant differences in performance on simulators versus performance on patients (P = .99), suggesting direct transfer of skills.
Learner knowledge, confidence, and satisfaction
Simulation-based educational intervention seemed to have favorable effects on learner knowledge, confidence (see Supplemental Digital Table 3 at http://links.lww.com/ACADMED/A56), and satisfaction. Moderate heterogeneity was observed (I2 = 41.2%, P = .17). Pooled random-effects SMD was 0.60 (95% CI 0.35–0.84, P < .001). Funnel plot was asymmetric, suggesting the possibility of publication bias (Begg P = .04).
For learner confidence, pooled random-effects SMD was 0.41 (95% CI 0.30–0.53) for studies with single-group pretest and posttest design and was 0.52 (95% CI 0.23–0.81) for studies with nonrandomized, two-group design (Figure 2, panel 2C). Statistical testing using the Begg was positive (Begg P = .03).
Ten studies reported measures of learner satisfaction post simulator training.11,24,25,27–29,31,33,37,38 Training was uniformly well received by learners. Lack of reported outcomes on satisfaction for control groups precluded pooling of results.
Patient clinical outcomes
Nine studies reported on patient outcomes, which included number of needle passes, arterial puncture, pneumothorax, and catheter-related infections (see Supplemental Digital Table 4 at http://links.lww.com/ACADMED/A56).
For studies of nonrandomized, two-group design, but not randomized controlled trials, simulation training was associated with significant decreases in average number of needle passes (SMD −0.58; 95% CI −0.96 to −0.20) and risk of pneumothorax (pooled risk ratio 0.62; 95% CI 0.40 to 0.97); see Figure 3. There was no significant decrease in risk of arterial puncture.
Outcomes of catheter-related infections were reported by four studies (see Supplemental Digital Table 4 at http://links.lww.com/ACADMED/A56).11,36,37,40 However, numerator and denominator for events were missing in one study, and that study was excluded from analysis.37 Simulation training was not associated with significant differences in catheter-related infection risk (Figure 3, panel 3D).
MERSQI scores for randomized studies (13; SD = 2) were not significantly different from scores for nonrandomized studies (14.9 ± 0.22; P = .16). Statistical testing using the Begg test found no evidence for funnel plot asymmetry in any of the patient outcomes (P > .05).
Discussion and Conclusions
Our systematic review and meta-analysis identified a number of publications supporting the value of simulation-based education for CVC. Benefits were seen in learner performance outcomes on simulators, learner knowledge, and confidence. Benefits were also seen in clinical outcomes such as decrease in the number of needle passes and decrease in the risk of pneumothoraces. These clinical benefits were demonstrated by studies of nonrandomized, two-group designs but were less convincing in studies that were randomized controlled trials. Last, regarding the risk of catheter-related infections, we found no statistically significant differences in study participants who received simulator training compared with those in control groups who did not.
To our knowledge, no previous systematic review on the effect of simulation-based education for CVC has been undertaken. A previous systematic review on surgical simulation concluded that benchtop simulation may be better than no training or standard training, but only four studies were included for that portion of the review.43 Our findings extend the work of others by providing current evidence pertaining specifically to the insertion of central venous catheters. Achieving competency in this procedure is a stated objective for a number of postgraduate training programs.44–47
The methodologies of the studies in the body of literature on which this meta-analysis is based have a number of limitations. First, the majority of the studies were observational. Pooled results are therefore subject to the effect of both measured and unmeasured confounders. Nonetheless, within the confines of the study design limitations, the mean MERSQI score for assessment of the quality of the studies included in this meta-analysis was quite high (12.6; SD = 2.3). To place this score in context, one may compare this score with a mean MERSQI score of 10.7 reported for a sample of manuscripts on medical education accepted for publication by one general medical journal.48
Second, statistical testing of funnel plots for asymmetry was significant in all three pooled learner outcomes, raising the possibility of publication bias. Although no significant asymmetry was detected for clinical outcomes, absence of significant asymmetry does not mean that publication bias was absent.49
Third, limited data are available on skill retention and direct skill transfer for CVC. Data from the surgical literature indicate that skills acquired by simulation-based education are transferable to the operative setting.50 Whether or not results are generalizable to a bedside procedure such as CVC is unknown.
Fourth, we were not able to isolate the effects of the technical aspects of simulation-based education from those of didactic teaching and other infection control cointerventions. Complications from central venous catheters arise beyond the act of insertion itself. Other aspects of CVC such as availability of proper equipment and supplies, catheter maintenance, and monitoring often require coordinated interprofessional team effort.51 Therefore, the extent of benefits gained from a simulation-based educational intervention needs to be placed within the context of other hospital-wide educational efforts aimed at decreasing complications.
Last, despite evidence that using ultrasound decreases complication rates in CVC52 and is becoming the standard of care,53,54 only 45% of studies reported the use of ultrasound in the educational intervention.
In addition, our review has limitations. First, despite work by others clearly demonstrating the effectiveness of providing feedback, repetitive practice, and providing learners with a range of task difficulty,55 we did not explore the effects of specific elements of curriculum design on outcomes. Our review suggests that whereas some of these elements were clearly included in the educational intervention (up to 45% of studies reported giving feedback), other features were rarely reported, such as presenting learners with a range of task difficulty. Second, some studies could not be included in the pooled analyses because of missing data, despite contacting the authors for more information. Third, unexplained heterogeneity was noted among several outcomes. Last, pooling effect sizes across study designs is problematic.17 We have therefore provided results for our meta-analyses, stratified by study design.
These limitations notwithstanding, this meta-analysis provides a detailed assessment of the effects of simulation-based education for CVC on learner and patient outcomes. Benefits seen in learner outcomes were consistent, with moderate effect sizes. In light of increasing concern regarding patient safety issues,56 we recommend that the implementation of simulation-based educational programs become a strategic priority for educating learners who insert central venous catheters. Assessing for the effectiveness of elements of instructional designs relating to learners, instructors, simulators, and environment was beyond the scope of our review. For future studies, we recommend that the academic research community focus its attention on the following questions:
- What is the effect of simulation-based education on clinical outcomes?
- What is the effect of specific elements of instructional design such as mastery learning,25,40–42 simulator fidelity, and teaching environment on competency?
- What is the role of simulation in competency assessment?
- What is the effect of simulation-based education on skill retention and skill transfer?
In conclusion, results from our meta-analysis support the use of simulation in the education of central venous catheter insertion. Simulation-based teaching for CVC should be made available for learners striving to attain competency in this procedure.
This study was funded in part by the Departments of Medicine and Surgery Research Development Fund from the University of Calgary. Paul Ronksley received funding from the Frederick Banting and Charles Best Canada Graduate Scholarship from the Canadian Institutes of Health Research. Dr. Ghali received funding from the Government of Canada Research Chair in Health Services Research and Senior Health Scholar award from Alberta Innovates-Health Solutions. The funding agencies had no role in the design and conduct of this study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, and approval of the manuscript.
1 Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282:861–866.
2 Okuda Y, Bond W, Bonfante G, et al. National growth in simulation training within emergency medicine residency programs, 2003–2008. Acad Emerg Med. 2008;15:1113–1116.
3 Ogden PE, Cobbs LS, Howell MR, Sibbitt SJ, DiPette DJ. Clinical simulation: Importance to the internal medicine educational mission. Am J Med. 2007;120:820–824.
4 Weinberg ER, Auerbach MA, Shah NB. The use of simulation for pediatric training and assessment. Curr Opin Pediatr. 2009;21:282–287.
5 Kneebone R. Simulation in surgical training: Educational issues and practical implications. Med Educ. 2003;37:267–277.
6 Matveevskii AS, Gravenstein N. Role of simulators, educational programs, and nontechnical skills in anesthesia resident selection, education, and competency assessment. J Crit Care. 2008;23:167–172.
7 Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein catheterization. Failure and complication rates by three percutaneous approaches. Arch Intern Med. 1986;146:259–261.
8 Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70–S81. http://journals.lww.com/academicmedicine/Fulltext/2004/10001/Deliberate_Practice_and_the_Acquisition_and.22.aspx
. Accessed May 8, 2011.
9 McGee DC, Gould MK. Preventing complications of central venous catheterization. N Engl J Med. 2003;348:1123–1133.
10 Kurrek MM, Devitt JH. The cost for construction and operation of a simulation centre. Can J Anaesth. 1997;44:1191–1195.
11 Sherertz RJ, Ely EW, Westbrook DM, et al. Education of physicians-in-training can decrease the risk for vascular catheter infection. Ann Intern Med. 2000;132:641–648.
12 Cohen ER, Feinglass J, Barsuk JH, et al. Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc. 2010;5:98–102.
13 Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med. 2009;151:264–269.
14 Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–1009.
15 Hedges L, Olkin I, eds. Statistical Methods for Meta-Analysis. San Diego, Calif: Academic Press; 1985.
16 Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to Meta-Analysis. Chichester, West Sussex, UK: John Wiley & Sons, Ltd; 2009.
17 Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, West Sussex, UK: John Wiley & Sons, Ltd; 2008.
18 Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York, NY: Academic Press; 1988.
19 Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–560.
20 DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–188.
21 Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–1101.
22 Guyatt G, Rennie D. User's Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, Ill: American Medical Association Press; 2002.
23 Wayne DB, Barsuk JH, Cohen E, McGaghie WC. Do baseline data influence standard setting for a clinical skills examination? Acad Med. 2007;82(10 suppl):S105–S108. http://journals.lww.com/academicmedicine/Fulltext/2007/10001/Do_Baseline_Data_Influence_Standard_Setting_for_a.28.aspx
. Accessed May 8, 2011.
24 Liachopoulou AP, Synodinou-Kamilou EE, Deligiannidi PG, Giannakopoulou M, Birbas KN. Nurses' training and confidence on deep venous catheterization. Technol Health Care. 2008;16:159–169.
25 Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70–76.
26 Carvalho P. Early introduction to central line placement: A curriculum for medical students. Med Educ. 2007;41:1098–1099.
27 Britt RC, Reed SF, Britt LD. Central line simulation: A new training algorithm. Am Surg. 2007;73:680–682.
28 Lee AC, Thompson C, Frank J, et al. Effectiveness of a novel training program for emergency medicine residents in ultrasound-guided insertion of central venous catheters. CJEM. 2009;11:343–348.
29 Millington SJ, Wong RY, Kassen BO, Roberts JM, Ma IW. Improving internal medicine residents' performance, knowledge, and confidence in central venous catheterization using simulators. J Hosp Med. 2009;4:410–416.
30 Sanchez LD, Delapena J, Kelly SP, Ban K, Pini R, Perna AM. Procedure lab used to improve confidence in the performance of rarely performed procedures. Eur J Emerg Med. 2006;13:29–31.
31 Nip IL, Haruno MM. A systematic approach to teaching insertion of a central venous line. Acad Med. 2000;75:552. http://journals.lww.com/academicmedicine/Fulltext/2000/05000/A_Systematic_Approach_to_Teaching_Insertion_of_a.83.aspx
. Accessed May 8, 2011.
32 Macnab AJ, Macnab M. Teaching pediatric procedures: The Vancouver model for instructing Seldinger's technique of central venous access via the femoral vein. Pediatrics. 1999;103:E8.
33 Ault MJ, Rosen BT, Ault B. The use of tissue models for vascular access training. Phase I of the procedural patient safety initiative. J Gen Intern Med. 2006;21:514–517.
34 Rosen BT, Uddin PQ, Harrington AR, Ault BW, Ault MJ. Does personalized vascular access training on a nonhuman tissue model allow for learning and retention of central line placement skills? Phase II of the procedural patient safety initiative (PPSI-II). J Hosp Med. 2009;4:423–429.
35 Velmahos GC, Toutouzas KG, Sillin LF, et al. Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. Am J Surg. 2004;187:114–119.
36 Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central line simulation before the ICU experience. Am J Surg. 2009;197:533–536.
37 Miranda JA, Trick WE, Evans AT, Charles-Damte M, Reilly BM, Clarke P. Firm-based trial to improve central venous catheter insertion practices. J Hosp Med. 2007;2:135–142.
38 Lenhard A, Moallem M, Marrie RA, Becker J, Garland A. An intervention to improve procedure education for internal medicine residents. J Gen Intern Med. 2008;23:288–293.
39 Martin M, Scalabrini B, Rioux A, Xhignesse MA. Training fourth-year medical students in critical invasive skills improves subsequent patient safety. Am Surg. 2003;69:437–440.
40 Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med. 2009;169:1420–1423.
41 Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397–403.
42 Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697–2701.
43 Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation: A systematic review. Ann Surg. 2006;243:291–300.
44 Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Critical Care Medicine. http://acgme.net/acWebsite/downloads/RRC_progReq/142pr707_ims.pdf
. Accessed May 10, 2011.
45 Joint Royal Colleges of Physicians Training Board, London. Specialty Training Curriculum for General Internal Medicine. http://www.jrcptb.org.uk/specialties/ST3-SpR/Documents/2009%20GIM%20curriculum.PDF
. Accessed May 10, 2011.
46 Royal College of Physicians and Surgeons of Canada. Objectives of Training in Emergency Medicine. http://rcpsc.medical.org/residency/certification/objectives/emergmed_e.pdf
. Accessed May 10, 2011.
47 Royal College of Physicians and Surgeons of Canada. Objectives of Training in the Specialty of General Surgery. http://rcpsc.medical.org/residency/certification/objectives/gen_surg_e.pdf
. Accessed May 10, 2011.
48 Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for medical education research study quality instrument scores: Quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med. 2008;23:903–907.
49 Ioannidis JP, Trikalinos TA. The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey. CMAJ. 2007;176:1091–1096.
50 Sturm LP, Windsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ. A systematic review of skills transfer after surgical simulation training. Ann Surg. 2008;248:166–179.
51 Berenholtz SM, Pronovost PJ, Lipsett PA, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32:2014–2020.
52 Hind D, Calvert N, McWilliams R, et al. Ultrasonic locating devices for central venous cannulation: Meta-analysis. BMJ. 2003;327:361.
53 Feller-Kopman D. Ultrasound-guided central venous catheter placement: The new standard of care? Crit Care Med. 2005;33:1875–1877.
54 American College of Surgeons. Revised Statement on Recommendations for Use of Real-time Ultrasound Guidance for Placement of Central Venous Catheters. http://www.facs.org/fellows_info/statements/st-60.html
. Accessed May 10, 2011.
55 Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach. 2005;27:10–28.
56 Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: An ethical imperative. Acad Med. 2003;78:783–788. http://journals.lww.com/academicmedicine/Fulltext/2003/08000/Simulation_Based_Medical_Education__An_Ethical.6.aspx
. Accessed May 8, 2011.