Journal Logo

Technical Reports

Effectiveness of Flow Volume Measurement Training Using a Custom-Made Doppler Flow Simulator

Lee, Hyung Seok MD; Park, Pyoungju RN; Han, Sohee RN; Joo, Narae MD; Song, Young Rim MD; Kim, Jwa Kyung MD; Kim, Cheolsu MD; Kim, Hyung Jik MD; Kim, Sung Gyun MD

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: February 2021 - Volume 16 - Issue 1 - p 73-77
doi: 10.1097/SIH.0000000000000469
  • Free

Abstract

An arteriovenous (AV) access is a lifeline for hemodialysis (HD) patients, and ultrasound (US) surveillance plays an important role in AV access management. The current vascular access guidelines recommend that dialysis staff monitor the flow volume (FV) of the access1–5; duplex ultrasound (DUS) is a noninvasive and powerful method of assessing the FV.6–10 The FV measurement can provide useful information on AV fistula maturation, dysfunction, and the outcomes of surgical or interventional treatments.11 However, the disadvantage of DUS is that the results depend on the experience of the sonographers. Accurate and reliable assessment of the access flow rate using DUS requires substantial training and experience; moreover, training with a simulator should be conducted before a novice examiner performs DUS on actual patients, although DUS is a safe and noninvasive procedure. This article aims to examine the effects on FV assessments performed by dialysis staff members, after training with a custom-made DUS flow simulator.

METHODS

This study was reviewed and approved by the Hallym University Sacred Heart Hospital Institutional Review Board, and written consent was received from all of the participants in this investigation (IRB No. HALLYM 2018-08-027-001).

Materials and Assembly of the Simulator and Phantom

The materials used for the simulator and phantoms are presented in Figure 1. To assemble the simulator, the following procedure was followed, as detailed in a previous report12: (a) the 24-Fr enema tube was immersed in the plastic container filled with keyboard cleaning gel. (b) Approximately 1.8 to 2.0 g of freeze-dried instant coffee granules were dissolved in 500 mL of 0.9% saline. (c) The continuous renal replacement therapy (CRRT) machine was primed in the slow continuous ultrafiltration mode after mounting the dialyzer set. (d) Manual priming was performed with the coffee solution after connecting the blood lines. (e) The CRRT machine was operated in the slow continuous ultrafiltration mode. (f) A duplex Doppler ultrasound (DUS; LOGIQ P5, GE Healthcare) was performed after applying acoustic gel on the cleaning gel.

F1
FIGURE 1:
Simulator materials and DUS image of phantoms. A, Materials for Doppler flow rate simulator using a CRRT machine. B, Materials for Doppler flow test fluid. C, Materials for vascular phantom. D, Triplex Doppler image of the Doppler test fluid and vascular phantom.

The fully assembled Doppler flow simulator is shown in Figure 2.

F2
FIGURE 2:
A, Assembly of Doppler flow rate simulator using CRRT machine. B, Training in FV measurement by Duplex ultrasound using the simulator and phantoms.

Study Design

Twelve dialysis nurses, with no previous experience performing Doppler flow rate examinations, volunteered to participate as trainees in this study. After their enrollment, the trainees received 3 days of instruction on the basic principles of DUS and methods of FV measurement. After the instruction, the trainees took 3 measurements of the FV on the AV access of an HD patient; the mean value and standard deviation (SD) of the measured values were compared with the reference value, which was measured by a registered vascular technologist (RVT) with more than 10 years of DUS experience. The trainees then went through a 3-day training course on measuring the FV using the simulator and phantom. During the simulator training, the effects of altering the Doppler angle of incidence, vessel diameter, degree of saturation in the spectral window, sample volume size adjustment, and Doppler frequency were simulated using the phantom for flow rate measurement. After the simulation training, the trainees took another 3 measurements of the FV of the AV access; the mean value and SD of the measured values were compared with the reference value, obtained by the experienced RVT. The differences between the measured FV and reference value, as well as the differences in the objective structured assessment of technical skills (OSATS; Table 1) before and after the simulation training, were analyzed (Fig. 3). The experienced RVT, acting as a supervisor, scored the volunteers on a scale of 1 to 3, according to the level of their psychomotor skills, with 1 being “poor” (cognitive stage), 2 being “fair” (associative stage), and 3 being “good” (autonomous stage).13 The total OSATS scores ranged from 33 to 99; trainees with a total score higher than 60 regarded as candidates to start DUS on their patients.

TABLE 1 - Objective Structured Assessment Technical Skills Checklist
Critical Procedures Performance
Section 1. Preparation
 Choose adequate probe and frequency, Hz 1 2 3
 Arrange patient position 1 2 3
 Adjust patient arm location 1 2 3
Section 2. B mode
 Understand probe orientation 1 2 3
 Ensure adequate image depth 1 2 3
 Set focus position and number 1 2 3
 Adjust gain 1 2 3
 Adjust dynamic range 1 2 3
 Adjust ultrasound beam perpendicular to the vessel wall 1 2 3
 Avoid pressing down the vein by probe weight 1 2 3
 Assess the vessel diameter 1 2 3
 Image cross-sectional plane 1 2 3
 Image longitudinal plane 1 2 3
Section 3. Color Doppler mode
 Ensure adequate image depth 1 2 3
 Place region of interest in the middle of screen 1 2 3
 Adjust color box size 1 2 3
 Adjust Doppler angle 1 2 3
 Adjust pulse repetition frequency 1 2 3
 Understand the flow direction 1 2 3
 Deal with artifacts 1 2 3
 Understand color threshold 1 2 3
 Ensure adequate saturation of vessel lumen 1 2 3
 Apply power Doppler and directional power Doppler scan 1 2 3
Section 4. Pulsed wave Doppler mode
 Ensure adequate image depth 1 2 3
 Adjust Doppler angle 1 2 3
 Adjust pulse repetition frequency 1 2 3
 Interrogate the sample volume in the middle of lumen 1 2 3
 Measure accurate vessel diameter 1 2 3
 Maintain introducer (cursor) parallel to the blood flow 1 2 3
 Maintain steady probe location during flow rate assessment 1 2 3
 Adjust sample volume size 1 2 3
 Deal with aliasing 1 2 3
 Control background noise in spectral waveform window 1 2 3
Total score
Scoring: 1 = poor, 2 = fair, 3 = good, total possible = 99.
Supervisor: _______________________.

F3
FIGURE 3:
Diagram of education and training plan.

After the second measurement test, postsimulation training, the volunteers were asked to fill out an anonymous survey on the helpfulness of the simulation training using a 5-point Likert scale, the results of which were reported as the mean and SD. We performed statistical analyses using the SPSS software (version 9.4; SAS Institute, Inc, Cary, NC). We present categorical variables in terms of their frequency and percentage and continuous variables in terms of the mean and SD; comparisons between the values before and after simulation training were conducted via paired t tests.

RESULTS

The difference between the reference value and mean values of the measurements performed by the trainees is designated as the measurement error, which indicates the degree of accuracy of the FV measurements. The SD and coefficient of variation (CV) of the 3 measurement values indicate the degree of reproducibility of the FV measurements.

Using a paired t test, the change in the mean value of the measurement errors in the FV assessment at the brachial artery, before and after training with the Doppler simulator, was analyzed. The measurement errors before and after the simulation training were 17.9 ± 11.8% (−131.6 ± 86.1 mL/min) and 8.6 ± 5.5% (−62.5 ± 48.7 mL/min), respectively [95% confidence interval (CI) = 3.7–14.8, P = 0.003]. The mean value of the difference from the reference value decreased by 69.0 mL/min after the simulator training (95% CI = 30.0 to 108.0, P = 0.002), as shown in Figure 4.

F4
FIGURE 4:
A, Change in the mean values of the FV measured by the trainees before and after simulation training (red dots, reference values; PRE, before simulation training; POST, after simulation training). B, Changes in the mean values of the FV measured by the 12 individual trainees before and after simulation training (zero baseline, the difference between the mean values measured by trainees and the reference value; PRE, before simulation training; POST, after simulation training).

The change in the SD of the FV values measured by trainees, before and after training with the Doppler simulator, was analyzed via a paired t test. The SDs of the FV values before and after the simulation training were 96.9 ± 57.0 and 47.0 ± 27.3 mL/min, respectively. The SD of the FV values was improved by 49.9 mL/min after training with the simulator (95% CI = 7.9 to 91.8, P = 0.023), as shown in Figure 5. The CV before and after the simulation training was 15.0 ± 7.3% and 5.8 ± 3.4%, respectively (95% CI = 3.6 to 14.7, P = 0.003).

F5
FIGURE 5:
A, Change in the SD of the FV values measured by the trainees before and after simulation training (PRE, before simulation training; POST, after simulation training). B, Changes in the SD of the FV values measured by the 12 individual trainees before and after simulation training (PRE, before simulation training; POST, after simulation training).

The mean value of the OSATS scores improved from 56.42 to 68.67 (95% CI = −15.6 to −8.8, P < 0.001). The result of the Likert scale (5 = strongly agree) was 4.58 ± 0.49.

DISCUSSION

The primary object of this study was to evaluate the effect of simulation training with a custom-made DUS flow simulator on dialysis staff making FV assessments. We hypothesized that simulation training using a CRRT machine, vascular phantom, and Doppler test fluid, which are all readily available materials in a dialysis center, would help improve the psychomotor skills of the 12 dialysis staff (who volunteered for this study) performing DUS examinations on actual patients. We previously reported that this custom-made simulator not only has a simple and affordable construction method but also provides certain advantages, such as good quality pulsatile waveforms and the prevention of air artifacts by trapping air bubbles with the venous drip chamber of the CRRT machine.12

The results of our investigation showed decreased CV and measurement errors after the simulation training, which indicates that the accuracy and reproducibility of the FV measurements had improved. The reproducibility is an indicator of the reliability, which has an effect on the intraobserver and interobserver variability.14

The CV was improved from 15% to 5.8% after the simulation training, which was comparable with the 12% of the previous report.15

Although the overall accuracy and reliability of the FV measurements significantly improved after the simulation training, some of the participants presented less accurate and reliable results after simulation training. In the comparison of the measurement errors, one participant underestimated the vessel diameter because of an ultrasound artifact, which resulted in an inaccurate FV assessment in the postsimulation training test. In addition, although the most trainees presented a decreased or similar SD of the measured FV, 2 participants documented an increased SD after the simulation training because of the instability of the transducer positioning on the targeted vessel. These findings indicate that more thorough trainings on the recognition of artifacts and the maintenance of transducer stability during the pulsed Doppler examination are necessary during simulation training for the novice examiners.

The other common mistakes made by the trainees when measuring the FV in the pulse wave Doppler mode included not imaging the vascular lumen as an accurate longitudinal plane, inappropriate adjustment of the sample volume, and erroneous interrogation of the Doppler incidence angle; in most cases, these errors resulted in an underestimation of the actual FV, which could lead to systematic errors in FV assessment. The trainees were able to simulate the causes of errors with the Doppler flow simulator and understand their effects on the FV assessment. Therefore, by comparing the values measured by the trainees to the actual values, trainees were able to avoid repeating the mistakes when taking Doppler flow rate measurements. This investigation demonstrated that theoretical knowledge alone was not sufficient to assess flow rates. Because both knowledge and technical skills are required to accurately perform DUS, the advancement of psychomotor skills is important before performing DUS examination on actual patients.

Although the ultrasound dilution technique regarded as the criterion standard of flow measurement methods, it has a limitation that makes it impossible to use for AV access in predialysis patients.16 Doppler ultrasound is the only noninvasive method available to measure the blood FV of the AV access in predialysis patients. However, performing accurate and reliable DUS FV measurements requires substantial experience and training. Clinicians often train to take FV measurements with a Doppler flow simulator; however, it is cost prohibitive for most clinical practices in Korea. Novice sonographers usually perform blood FV measurements on volunteers or patients during their training period because of the difficulty in accessing commercial Doppler flow simulators. However, it is recommended that one obtain sufficient training using simulators before performing diagnostic procedures on actual patients, whenever possible. Training dialysis staff in DUS flow rate assessments would be helpful in the early diagnosis of AV access dysfunction and the maintenance of durable vascular access.

Despite the fact that current computational models are accurate and repeatable, they require a computer-based system, software, and mannequins or phantoms, each bringing additional costs.17 Recent studies using computational flow dynamics systems have reported a mean deviation from actual peak systolic velocity of 7.8% to 10.0%. In this study, the mean difference in FV from the reference value was 8.6% after the simulation training, which was comparable with that of the computer-based simulator training.17,18

This study was limited in that it was a single-center study and the number of participants was small. In addition, it is possible that the outcomes of the measurements have improved because the trainees became more familiar with measuring the FV by the time they took the second blood flow test. However, all trainees agreed that the training with the proposed simulator helped understand how to better assess FV and the possible causes of the errors that might occur during FV measurement. The trainees also agreed that the simulation training enhanced their learning when compared with lectures alone.

Further research is necessary to assess the effectiveness of the simulation training on the measurement of flow parameters, such as peak systolic velocity, FV, and resistive index, and the cost-effectiveness in the various DUS flow simulators.

CONCLUSIONS

The DUS flow simulation training using a custom-made DUS flow simulator markedly improved the accuracy and reliability of AV access FV assessment. It is a promising method of effectively training dialysis staff on DUS flow rate measurements and it may be helpful in improving their ability of monitoring and surveillance on AV access.

REFERENCES

1. Vascular Access 2006 Work Group. Clinical practice guidelines for vascular access. Am J Kidney Dis 2006;48(Suppl 1):S176–S247.
2. Tordoir J, Canaud B, Haage P, et al. EBPG on vascular access. Nephrol Dial Transplant 2007;22(suppl 2):ii88–ii117.
3. Kumwenda M, Mitra S, Reid C. Clinical practice guideline: vascular access for haemodialysis-UK renal association. 2015:1–26.
4. Jindal K, Chan CT, Deziel C, et al. Hemodialysis clinical practice guidelines for the Canadian Society of Nephrology. J Am Soc Nephrol 2006;17:S1–S27.
5. Schmidli J, Widmer MK, Basile C, et al. Editor's choice - vascular access: 2018 Clinical Practice Guidelines of the European Society for Vascular Surgery (ESVS). Eur J Vasc Endovasc Surg 2018;55(6):757–818.
6. Guedes Marques M, Ibeas J, Botelho C, Maia P, Ponce P. Doppler ultrasound: a powerful tool for vascular access surveillance. Semin Dial 2015;28:206–210.
7. da Fonseca Junior JH, Pitta GB, Miranda Júnior F. Accuracy of Doppler ultrasonography in the evaluation of hemodialysis arteriovenous fistula maturity. Rev Col Bras Cir 2015;42(3):138–142.
8. Robbin ML, Chamberlain NE, Lockhart ME, et al. Hemodialysis arteriovenous fistula maturity: US evaluation. Radiology 2002;225(1):59–64.
9. Lomonte C, Meola M, Petrucci I, Casucci F, Basile C. The key role of color Doppler ultrasound in the work-up of hemodialysis vascular access. Semin Dial 2015;28(2):211–215.
10. van Hooland S, Malik J. Hemodialysis vascular access ultrasonography: tips, tricks, pitfalls and a quiz. J Vasc Access 2010;11(4):255–262.
11. Zamboli P, Fiorini F, D'Amelio A, Fatuzzo P, Granata A. Color Doppler ultrasound and arteriovenous fistulas for hemodialysis. J Ultrasound 2014;17(4):253–263.
12. Lee HS, Park P, Han S, et al. Custom-made Doppler ultrasound flow simulator for dialysis access using continuous renal replacement therapy machine. J Vasc Access 2019;20(6):701–705.
13. Fitts PM, Posner MI. Human Performance. Belmont, CA: Brooks/Cole Pub. Co; 1967.
14. Mikkonen RH, Kreula JM, Virkkunen PJ. Reproducibility of Doppler ultrasound measurements. Acta Radiol 1996;37(4):545–550.
15. Oates CP, Williams ED, McHugh MI. The use of a Diasonics DRF400 duplex ultrasound scanner to measure volume flow in arterio-venous fistulae in patients undergoing haemodialysis: an analysis of measurement uncertainties. Ultrasound Med Biol 1990;16(6):571–579.
16. Tessitore N, Bedogna V, Gammaro L, et al. Diagnostic accuracy of ultrasound dilution access blood flow measurement in detecting stenosis and predicting thrombosis in native forearm arteriovenous fistulae for hemodialysis. Am J Kidney Dis 2003;42(2):331–341.
17. Sheehan FH, Zierler RE. Simulation for competency assessment in vascular and cardiac ultrasound. Vasc Med 2018;23(2):172–180.
18. Leotta DF, Zierler RE, Sansom K, Aliseda A, Anderson MD, Sheehan FH. Evaluation of examiner performance using a duplex ultrasound simulator: flow velocity measurements in dialysis access fistula models. Ultrasound Med Biol 2018;44(8):1712–1720.
Keywords:

Hemodialysis; ultrasonography; Doppler ultrasound; arteriovenous fistula; simulation training

Copyright © 2020 Society for Simulation in Healthcare