FDA Benchmark Medical Device Flow Models for CFD Validation : ASAIO Journal

Secondary Logo

Journal Logo

Biomedical Engineering

FDA Benchmark Medical Device Flow Models for CFD Validation

Malinauskas, Richard A.*; Hariharan, Prasanna*; Day, Steven W.; Herbertson, Luke H.*; Buesen, Martin; Steinseifer, Ulrich; Aycock, Kenneth I.§; Good, Bryan C.§; Deutsch, Steven§; Manning, Keefe B.§; Craven, Brent A.*

Author Information
ASAIO Journal 63(2):p 150-160, March/April 2017. | DOI: 10.1097/MAT.0000000000000499


Computational fluid dynamics (CFD) is increasingly being used to develop blood-contacting medical devices. However, the lack of standardized methods for validating CFD simulations and blood damage predictions limits its use in the safety evaluation of devices. Through a U.S. Food and Drug Administration (FDA) initiative, two benchmark models of typical device flow geometries (nozzle and centrifugal blood pump) were tested in multiple laboratories to provide experimental velocities, pressures, and hemolysis data to support CFD validation. In addition, computational simulations were performed by more than 20 independent groups to assess current CFD techniques. The primary goal of this article is to summarize the FDA initiative and to report recent findings from the benchmark blood pump model study. Discrepancies between CFD predicted velocities and those measured using particle image velocimetry most often occurred in regions of flow separation (e.g., downstream of the nozzle throat, and in the pump exit diffuser). For the six pump test conditions, 57% of the CFD predictions of pressure head were within one standard deviation of the mean measured values. Notably, only 37% of all CFD submissions contained hemolysis predictions. This project aided in the development of an FDA Guidance Document on factors to consider when reporting computational studies in medical device regulatory submissions. There is an accompanying podcast available for this article. Please visit the journal’s Web site (www.asaiojournal.com) to listen.

Computational fluid dynamics (CFD) can be a valuable tool for characterizing flow fields by predicting velocities, pressures, and shear stresses using numerical techniques. Over the past 50 years, CFD applications have expanded from examining flows around airfoils and automobiles to the development and evaluation of blood-contacting medical devices.1,2 The advantages of using CFD in device design are that simulations can 1) provide insight into device performance without having to produce costly prototypes, 2) provide data on critical regions that are not easily accessible for measurement or visualization (e.g., small gap regions between a blood pump blade tip and the housing, within the hinge of a mechanical heart valve), and 3) predict difficult-to-measure physical quantities which may impact blood damage, such as wall shear stress and particle residence time.3,4

Although the U.S. Food and Drug Administration (FDA) does not require CFD simulations in the evaluation of blood-contacting medical devices, international standards for heart valves5,6 and implantable circulatory support devices7 recognize that experimentally validated CFD simulations may be used to characterize the flow fields within and around these devices, and to assess their hemolytic and thrombogenic potentials. However, the ISO 14708-5 standard indicates that the use of CFD should be limited to the design stage and that it is more appropriate for evaluating relative design changes rather than assessing absolute quantities.7 In effect, the use of CFD as a regulatory tool for predicting absolute values of blood damage (e.g., hemolysis, thrombosis) as part of a safety evaluation of a medical device is not well established.

A key aspect for instilling confidence in the predictive capability of a CFD simulation is through verification and validation.8 Verification refers to assessing the accuracy of the numerical solutions of the governing equations and includes mesh design and sensitivity, spatial and temporal discretization accuracy, and convergence criteria. Validation refers to determining whether the simulation is an accurate representation of the real world that is being modeled.8 For blood-contacting medical devices, CFD validation has two important components: validation of the flow field, and validation of the predicted biological response. Flow field validation is often conducted by comparing the computational simulations with experimentally measured values of pressures and fluid velocities within devices or device models. Validation should consider the full operating range of the device because different types of CFD models may be required to resolve the flow depending on whether it is laminar, turbulent, or transitional.8 Other validation issues that need to be considered in the CFD analysis include how well the model input parameters (e.g., fluid properties, inlet and outlet boundary conditions) accurately mimic the validation experiments.9–14

Computational predictions of biological responses are based on flow field metrics (e.g., high shear stresses and exposure times, stagnant flow regions) which are believed to contribute to hemolysis, platelet activation, and thrombosis.1,2,11,15 Although many constitutive blood damage modeling equations have been proposed based on empirical data,4,15–17 there are no widely accepted techniques for accurately predicting absolute levels of hemolysis or thrombosis. Variants in computational hemolysis models include determining red blood cell damage based on different fluid stress-exposure time power-law models,15,16 strain-based cell models,2,18,19 and energy dissipation models.20 Modeling of thrombosis can be even more challenging as platelet activation and thrombosis initiation depend on multiscale linked hemodynamic and biochemical processes, such as platelet margination to vessel walls,21 and advection, diffusion, and surface reactions.2,22,23 As blood damage test data are usually only obtained in one laboratory, the results may only be applicable for the specific blood, test model, and test conditions used in the individual experiments. The lack of multi-laboratory-derived experimental blood damage data further hinders adequate validation of these computational models, and their use for regulatory evaluation.

Because CFD has the potential to impact medical device innovation and evaluation, the FDA initiated a project to assess the current status of CFD device modeling and to address the limitations which impede its utility as a regulatory tool.10 As validation of simulations is a major concern in the acceptance of CFD in regulatory practice, this project focused on creating two well-defined, “benchmark” flow models (a nozzle and a blood pump) that could be tested to establish reliable and publicly available experimental datasets. For each flow model, measurements of velocity, pressure, and hemolysis were acquired in up to three independent laboratories. To evaluate the current state of CFD use, computational studies open to anyone were performed on each model using FDA-specified test conditions. Particle image velocimetry (PIV), CFD, and hemolysis testing results from the first study on the nozzle model have previously been disseminated through a series of publications.9,10,24,25 The primary goal of this article is to summarize the FDA initiative and to report recent findings from the second benchmark study using the blood pump model.


The two models were designed to provide flow characteristics which are representative of those in medical devices (e.g., laminar, transitional, and turbulent flows; gradual and sudden changes in flow areas; flows around rotating components). Physical models were fabricated out of polished transparent acrylic for experimental testing.

Study 1: Nozzle Benchmark Model

For the first study, a bidirectional nozzle model was created in which the flow diameter changed either suddenly or gradually (Figure 1).9,10,24 Particle image velocimetry measurements were acquired in three laboratories using a Newtonian blood analog fluid which matched the refractive index of the nozzle models. The flow rate was adjusted so that the velocity field and pressure along the model could be measured at Reynolds numbers in the nozzle throat which corresponded to laminar (Re = 500), transitional (Re = 2000), and turbulent flow regimes (Re = 3500, 5000, 6500). Computational fluid dynamics simulations, conducted under the same flow conditions as the PIV experiments, were received from 28 different groups. Hemolysis testing was also conducted in three laboratories; however, the flow rates were increased so that measurable hemolysis data could be obtained. Specifically, the hemolysis tests were performed at blood flow rates of 5 and 6 L/min for the sudden contraction at the entrance (corresponding to average throat Reynolds numbers of 6650 and 8020, respectively), and at a blood flow rate of 6 L/min for the gradual contraction at the entrance (n = 26–36 repeats at each of the three test conditions).

Figure 1.:
FDA benchmark nozzle model with internal diameters and nozzle throat length identified.

Study 2: Blood Pump Benchmark Model

The second benchmark model was a centrifugal blood pump designed to have simple geometrical features and to operate over a wide range of flow and pressure conditions (Figure 2). The acrylic rotor (5.2 cm in diameter) had four filleted blades (3 mm tall and 3 mm wide) orthogonally positioned on a 4 mm thick rotor base, attached to a stainless steel shaft (3.2 mm in diameter). The shaft was sealed against blood leakage by using two spring-loaded, polymer-filled PTFE seals (Bal Seal Engineering, Foothill Ranch, CA) and grease (Molykote 111, Dow Corning, Midland, MI) recessed within the rear housing. Preliminary hemolysis testing was conducted to demonstrate that the shaft seal system was no different than if a saline-flush seal system was used. Using spacers, 1 mm clearance gaps were set between the housings and the front and back of the rotor during the PIV and hemolysis tests. Quality control measurements ensured that the physical dimensions were within 1% of the design specifications and surface finishes had Ra < 0.6 μm (Ra = 0.24 ± 0.23 μm, Contour GT-K 3D Optical Microscope, Bruker AXS, Madison, WI) for the blood-contacting pump components.

Figure 2.:
A: Assembly drawing for FDA benchmark pump model. B: Enlargement of rotor showing dimensions of rounded blade edges. C: Computer model of pump.

For both the PIV and hemolysis tests, the blood pump model was incorporated into a closed recirculating flow loop (Figure 3) in which the flow rate, pre- and postpump pressures, fluid temperature, and pump speed could be controlled and measured. All tubing in the loop was ½" ID PVC, except for a 3/8" ID section downstream of the pump outlet which was clamped to adjust the backpressure on the pump. To increase the sensitivity of the hemolysis tests, custom-designed blood bag reservoirs26 and flow-through heat exchangers were used so that the test loop could be operated with a minimum blood volume of only 240 ml.

Figure 3.:
Hemolysis test loop containing the following components (clockwise starting at the pump): FDA blood pump model, flow probe, pressure (P) transducer, clamp to control the backpressure, blood temperature thermistor, custom-made blood reservoir, heat exchanger, and prepump pressure/blood sampling/fill port.

To examine the effect of flow rate and rotational speed on hemolysis, six pump operating conditions, spanning flow rates of 2.5–7 L/min at pump speeds of 2500 and 3500 rpm (Table 1), were chosen for detailed analysis as reflected on the pressure versus flow rate hydrodynamic characterization curves (Figure 4A). Preliminary testing using prototype designs determined that outside of the six chosen operating conditions hemolysis was minimal (speed < 1800 rpm), or could potentially be caused by cavitation (flow rate > 7 L/min) or excessive tube clamping (backpressure > 350 mm Hg) in addition to the fluid forces within the blood pump.

Table 1.:
The Six Test Conditions Used to Evaluate the FDA Benchmark Blood Pump Model Using Hemolysis Testing and CFD Simulations
Figure 4.:
A: Pressure head versus flow rate characterization curves for the benchmark pump model using 36% hematocrit porcine blood at 25°C. The six test conditions used for hemolysis testing, PIV measurements, and CFD simulations are identified with numbered circles. B: Pressure head measured during hemolysis and PIV tests (mean ± SD) at the six pump test conditions compared to individual (black circles) and mean predictions (red circles) from the CFD simulations. The number of samples used to determine the mean values for each group are indicated in the legend. CFD, computational fluid dynamics; PIV, particle image velocimetry; SD, standard deviation.

Particle image velocimetry testing of the blood pump model.

Particle image velocimetry experiments were performed concurrently at three laboratories using a Newtonian blood analog fluid composed of 50–53% saturated aqueous sodium iodide solution, 16–17% glycerin, and 31–33% water, by weight. The dynamic viscosity of the fluid varied between 4 and 7 cP at 25°C, the density ranged from 1.600 to 1.750 g/ml, and the refractive index was 1.48–1.49 to match that of the acrylic models. Because of the difference in density and viscosity of the PIV fluid compared with blood, the flow rate and pump speed were scaled using the Reynolds number and flow coefficient (Equations 1 and 2).

where is the density and is the dynamic viscosity (measured on each test day), Ω is the angular pump speed (rad/s), D is the impeller diameter (5.2 cm), and Q is the flow rate.

The PIV measurement systems incorporated a double-pulsed Q-switched multimode Nd:YAG laser (λ = 532 nm) with a repetition rate of 8–15 Hz as the illumination source. Using a lens system, the laser beam was focused into a thin laser sheet (0.5–0.8 mm thick) for planar illumination of 10 µm fluorescent polymer spheres which acted as tracer particles for visualizing the flow (Polysciences, Inc., Warrington, PA). The specific gravity of the spheres was approximately 1.1, while that of the fluid was 1.6–1.8. Best practice procedures for performing PIV in the nozzle study were followed for analyzing the pump.24 Image capture was via a charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) camera connected to a computer. Particle image velocimetry measurements were made at 11 sampling planes covering the inlet, mid-blade, front and back rotor gaps, blade tips, cutwater, and the exit diffuser regions of the pump. The spatial resolution of the PIV velocity measurements depended on the required field of view in the sampling cross-sections. For example, a coarse resolution of 0.4–0.6 mm was used to capture images of an entire pump quadrant to measure the velocity field between the rotating blades. The PIV resolution was 0.28–0.33 mm for the exit diffuser region, whereas a higher resolution of 0.15 mm was used for resolving the large velocity gradients near the cutwater edge. The ensemble-averaged two-dimensional velocity field was obtained by averaging the cross-correlation results of 500–2500 pairs of images captured at the same rotor position (i.e., with one of the blades positioned 90° to the outlet). Fluid shear rates were calculated from the velocity fields. The pressures at the pump inlet and exit were also measured for comparison to the blood experiments and the CFD simulations.

Computational fluid dynamics simulations of the blood pump model.

As summarized in Table 2, 23 participants from five different countries submitted 24 simulations for the CFD pump challenge. The identity of each participant was removed from the submitted simulations before the analysis of the CFD results. Overall, 14 steady-state and 10 transient simulations were received. Steady-state simulations used a frozen rotor, multiple reference frame approach with the impeller in a fixed position. Transient simulations directly modeled the rotation of the impeller relative to the static pump housing using a sliding mesh approach. All but one of the participants used commercial CFD software. The most popular choice of turbulence model was the k–ω shear-stress transport model, followed by variants of the k-ε turbulence model (realizable, standard, RNG; Table 2). A majority of the CFD submissions (16 of 24) used a tetrahedral mesh topology, whereas the other participants performed simulations using either a hexahedral (4) or polyhedral (4) mesh. The three-dimensional volumetric pump meshes that were used by the participants ranged in size from relatively coarse (0.53 million computational cells) to extremely fine (76.5 million cells).

Table 2.:
Summary of CFD Participant Submissions with the Number of Submissions Indicated in Parentheses for Select Parameters

An analysis was performed to compare CFD results for field data and scalar quantities at the six pump-operating conditions. Field data included velocity, shear stress, and turbulent viscosity provided on defined cross-sectional planes through the volume of the pump and on various surfaces of the rotor, pump housing, and exit volute. For each sampling plane, the results from each CFD simulation were interpolated using a Kriging interpolation tool (Tecplot 360, Tecplot, Inc., Bellevue, WA) onto a common mesh having a resolution six times greater than any of the submitted data sets. Mesh generation software (Pointwise, v. 17.3, Pointwise, Inc., Fort Worth, TX) was used to create the common “interpolation meshes” with high-resolution wall-normal layers to resolve large, near-wall gradients.

Scalar quantities included inlet and outlet pressures, shaft torque, absolute plasma hemoglobin concentration (pfHb), and a relative hemolysis index (RHI). The relative hemolysis index was defined as the hemolysis predicted at each test condition normalized by the predicted value at condition 5. Condition 5 was used for normalization because preliminary testing showed that a measurable level of hemolysis would be present at this test point.

Hemolysis testing of the blood pump model.

The six pump-operating conditions were randomized for hemolysis testing at one laboratory. In all cases, two benchmark pump models and a clinical control pump were run simultaneously at the same flow rates and backpressures for comparison. Eight separate porcine blood pools were used to conduct 16 replicates for the blood pump model and 8 replicates for the control pump at each test condition. Each blood pool was obtained from two to three slaughterhouse animals or live donors and anticoagulated with acid citrate dextrose solution A (ACDA). Blood was filtered through a 75 μm polypropylene mesh, and the hematocrit was adjusted to 36% via hemoconcentration using a centrifuge or dilution with phosphate-buffered saline (PBS). After pre-rinsing each flow loop with PBS, 250 ml of blood was added and maintained at 25°C. Each experiment was run for a total of 120 minutes, and samples were drawn every 40 minutes. At each sampling time point, the test flow parameters (flow rate, shaft rpm, blood temperature, pre- and postpump pressure, and shaft seal gland temperature) were recorded, 1 ml of blood was drawn and discarded from the sampling port, and two 2 ml blood samples were pulled for analysis. After centrifugation, pfHb levels were determined using the Cripps method to quantify the hemolysis.27

Statistical Analysis

Statistical analysis results are reported for sample numbers (n) as mean ± standard deviation (SD), with variability defined by the coefficient of variation (i.e., SD/mean × 100%). For comparing the pressures and hemolysis levels among the six pump test conditions, analysis of variance (ANOVA) tests were performed with a Tukey posttest if a significant difference was identified. For data sets that did not pass the assumption of normality and equal variance, even after log transformation of the data, a nonparametric ANOVA test was performed. The Grubbs extreme studentized deviate test was used to determine outliers in the hemolysis results at each test condition.28


Study 1: Nozzle Benchmark Model

Particle image velocimetry measurements showed good agreement (<10% variability among the laboratories) when Re was either laminar (Re = 500) or turbulent (Re = 3500–6500). However, when the flow was transitional in the nozzle throat (Re = 2000), the downstream center-line velocity measurements had greater interlaboratory variability as they were especially sensitive to the inlet conditions.24

The PIV measurements of velocity were compared with 28 CFD simulations received for the nozzle.9,10 Participants were able to simulate the center-line velocity within the entrance tube and throat of the nozzle reasonably well, but pronounced discrepancies occurred because of flow separation in the downstream sudden expansion region (Figure 3 in Stewart et al.9), particularly for transitional and low Reynolds number turbulent flows (Re = 2000–3500).

Hemolysis testing showed comparable results between the three test laboratories (Figure 4A in Herbertson et al.25). Hemolysis was greatest for the “sudden contraction” nozzle configuration and at the highest flow rate of 6 L/min (Re = 8000). Because hemolysis was minimal below Re = 6500, blood damage testing was performed at different throat Reynolds numbers (Re ~6500, 8000) than for the CFD and PIV analyses (Re = 500–6500). Hence, the experimental hemolysis results could not be directly compared with the values predicted by the CFD participants for the nozzle model.

Study 2: Blood Pump Benchmark Model

Particle image velocimetry testing of the blood pump model.

Results of PIV testing were compiled from a total of four separate data sets received from three laboratories. At each test condition, the pressures measured during the PIV and hemolysis experiments (Figure 4B) were not significantly different from each other (ANOVA, p > 0.05).

For brevity, PIV and CFD velocity results are mainly presented at test condition 5 in a cross-sectional plane located 1.2 mm from the top surface of the impeller blades. This upper-blade plane allowed for examination of flow within the body of the pump and also coincided with the mid-axis plane for the exit diffuser region. The velocity magnitude inside the pump increased radially outward with a maximum velocity of 8.7 ± 0.5 m/s occurring at a radial location of r = 25.4 ± 0.75 mm and trailing behind the vertical blade tip (Figure 5). The corresponding blade velocity (calculated as 2πNr, where N is the 3500 rpm pump speed and r is the 25.4 mm radial location along the blade) was 9.3 m/s. Good agreement between the three laboratories was also observed for the velocity magnitude along a diagonal line (at 45°) through the first quadrant of the pump (Figure 6A), with variability for the four datasets of 5–25%.

Figure 5.:
Contour of PIV two-dimensional velocity magnitude for the upper-blade plane within the pump and expansion region of the exit volute (averaged from the results of four datasets from three laboratories) at pump condition 5. The average PIV image resolutions for the (1) blade-passage, (2) cutwater (dashed outline box), and (3) exit diffuser regions were 0.39, 0.15, and 0.27 mm, respectively. Solid outlines of the pump hub (centered at coordinates 0,0) and two orthogonal rotor blades are shown.
Figure 6.:
Profiles of two-dimensional velocity magnitude, |U xy|, at a pump condition of 6 L/min and 3500 rpm for the PIV data (A, B) and for 12 of the steady-state CFD submissions (C, D) at the same locations. Velocity magnitude extracted along a line between the two rotor blades (as shown in the inset in A) is plotted versus radius from the pump centerline for (A) PIV and (C) CFD. Velocity profiles in the diffuser extracted at x = 0.035 m (as shown in the inset in B) are plotted versus distance from the inner wall for (B) PIV and (D) CFD.

At test condition 5, the outlet jet in the exit diffuser was skewed toward the outer wall in all four data sets (Figures 5 and 6B). This asymmetry created a large recirculation region near the inner wall of the pump diffuser (see Video, Supplemental Digital Content, https://links.lww.com/ASAIO/A129, which demonstrates the exit jet and recirculating flow pattern). For flow rates above 4.5 L/min (i.e., conditions 4, 5, and 6), the exit jet angled toward the outer wall for both pump speeds (2500 and 3500 rpm). However, for Q = 2.5 L/min (test conditions 1 and 2), the jet was skewed toward the inner wall of the exit diffuser.

Computational fluid dynamics simulations of the blood pump model.

Where noted, results that were deemed to be unreliable due to suspected participant reporting errors were not included in the present analyses. For the CFD pressure head predictions (n = 19–21 at each test condition), 57% were within the one standard deviation bounds defined by the pressure measurements made during the hemolysis and PIV experiments at each test condition (Figure 4B). The variability in CFD predictions of pressure head was greatest for conditions 3–6.

For this summary, only the velocity results for 12 of the 14 submitted steady-state simulations are presented because of suspected reporting errors in two submissions. Comparison of the interpolated participant velocity field data in the blade passage plane for condition 5 showed that the steady-state CFD simulations qualitatively predicted similar flow patterns in the pump, but highly varied flow patterns in the diffuser (Figure 7). In most of the CFD submissions, the maximum velocities occurred around the front, outer, or trailing edge of the impeller tip with values slightly greater than the impeller tip speed (9.52 m/s, r = 26 mm). Subtle differences in the CFD-predicted flow patterns for a few participants included small regions of low-speed flow in the vicinity of the cutwater (Figure 7E, H, I, and K) and along the upper wall of the pump housing (Figure 7B and G).

Figure 7.:
Interpolated contours of three-dimensional velocity magnitude, |U|, for the upper-blade passage plane at pump condition 5 for 12 of the 14 steady-state CFD submissions. Example regions where flow varied between simulations are identified in panels A (zone 1: diffuser jet), B (zone 2: upper wall of housing), and E (zone 3: below the cutwater tip).

All participants predicted separated flow in the diffuser, but the location of the exit jet and recirculation zone varied. Three submissions (Figure 7A, K, and L) predicted that the flow attaches to the inner wall of the diffuser, whereas two participants (Figure 7C and F) predicted that the jet forms roughly in the center of the diffuser. The other seven steady-state CFD simulations shown in Figure 7 all predicted skewed flow toward the outer wall of the diffuser, which is more consistent with the PIV experimental observations (Figure 5).

Within the rotating region of the pump, the velocity magnitude predicted by the steady-state CFD submissions compared reasonably well with the PIV data (Figure 6A and C). At x = 0.035 m within the diffuser, however, only two of the steady-state CFD submissions accurately predicted the location and magnitude of the detached jet when compared with the experiments (Figure 6B and D). A preliminary analysis indicates that this may be at least partially due to the same choice of turbulence model in both submissions (realizable kε).

Hemolysis testing of the blood pump model.

The blood used in the tests had an adjusted hematocrit of 36 ± 1%, a viscosity of 3.4 ± 0.5 cP (measured at a shear rate of 500 s−1 at 25°C), and a density of 1.03 ± 0.01 g/ml, which matched the blood parameters used by the CFD modelers. During the 2 hour hemolysis tests, the loop blood temperature remained at 25 ± 1°C, and the temperature of the shaft blood seal gland stabilized at 27–29°C.

Hemolysis generation occurred at a constant rate over time (i.e., for pfHb vs. time, 88% of all hemolysis tests had r2 > 0.95). Figure 8 shows the in vitro pfHb results for the blood pump model at the six operating points. Of the 96 total pump hemolysis tests (six test conditions with 16 repeats each), a total of four single outliers were identified at four separate test conditions. Hemolysis levels were statistically different between the two extreme test conditions and most of the other test conditions (by ANOVA test after removal of outliers). Specifically, condition 1 was statistically different from conditions 3–6, and condition 6 was statistically different from all of the other conditions. Levels of hemolysis in the clinical control blood pump loops (data not shown) were significantly lower at each test condition as compared to the benchmark blood pump model (ANOVA, p < 0.05).

Figure 8.:
In vitro hemolysis results for the blood pump model at the six operating points (n = 16 replicates). Circles indicate four outliers removed from the statistical analysis. Condition 1 was statistically different from conditions 3–6, and condition 6 was different from test conditions 1–5 (p < 0.05).

Computational fluid dynamics predictions of hemolysis were reported only in 8 of the 24 blood pump submissions. Because of suspected reporting errors by three participants, absolute levels of generated plasma hemoglobin for direct comparison to the experimental hemolysis results are not presented. Instead, the participant-provided relative hemolysis values are compared with the experimentally calculated RHI results. As shown in Figure 9, the computationally derived values of RHI from the eight participants vary widely, even though seven of the eight participants used a shear stress–based power-law model.4,16,29

Figure 9.:
Relative hemolysis index (hemolysis normalized to the results from test condition 5) for the six pump test conditions for the CFD submissions and hemolysis experiments. At each test condition, results from the eight CFD participants that submitted hemolysis data are followed by the RHI value (mean + SD) obtained in the hemolysis experiments (ninth bar labeled with an “E”). RHI, relative hemolysis index; SD, standard deviation.


Computational fluid dynamics modeling is a valuable tool in the design stages of medical device development, but the lack of standard procedures for demonstrating the validity of complex flow field simulations and blood damage predictions limits its use in risk-based evaluations. This article provides an overview of our collaborative research efforts to contribute publicly available data to help improve the use of CFD as a credible tool in the safety evaluation of blood-contacting medical devices. The main goals of our project are to: 1) establish a set of benchmark models that are representative of cardiovascular medical devices and provide experimental data (for velocity, pressure, shear stresses, and hemolysis) which could be used for validating CFD practices; 2) understand current practices of applying CFD to medical devices by facilitating open CFD challenges; 3) collaboratively develop standards for applying CFD in the regulatory evaluation process for medical devices; and 4) promote the further development of methods to reliably predict blood damage (e.g., hemolysis, thrombosis, platelet activation).

Two benchmark flow models, a nozzle and a centrifugal blood pump, were characterized under varying flow conditions using PIV analysis (to measure the velocity, shear stress, and pressure fields) and hemolysis testing (to determine blood damage potential) for comparison to computational predictions. Experiments were performed in multiple laboratories to establish ranges of measurement error and to make the results as robust as possible. All details about the benchmark models, the CFD simulation challenges, and the available experimental results can be found at https://fdacfd.nci.nih.gov.30

Analysis of the nozzle benchmark model revealed that CFD predictions of the velocity field in even a simple, steady-flow model may present challenges, as many of the simulations were unable to accurately predict velocities and shear stresses in the recirculation zones downstream of the sudden expansion.9 Moreover, this study also demonstrated the modeling issues associated with accurately predicting transition from laminar to turbulent flow, which often occurs in medical devices.4,10,11 Based on the nozzle analysis, a list of 21 “CFD Best Practice” guidelines were developed that include justifying the choice of a turbulence model and its parameters, identifying and quantifying the necessary boundary and initial conditions, confirming numerical convergence, performing a grid refinement study, and validating the simulation to reliable experimental data obtained in similar types of models using quantitative measures.9,10 These and other important considerations for verification and validation of computational modeling used to evaluate medical devices are described in an FDA Guidance Document12 and a draft American Society of Mechanical Engineers (ASME) standard.14

The significance of this project is further exemplified by the use of our interlaboratory PIV data for the nozzle model by multiple groups as a benchmark data set for validating and improving in-house, open-source, and commercial CFD solvers. In the laminar flow regime, White et al.31 used this model to assess the Lattice Boltzmann method, and Trias et al.32 concluded that treating the blood as Newtonian instead of a non-Newtonian fluid led to an underestimation of the hemolysis levels. Several studies have also used the nozzle PIV data to validate advanced turbulence models. Fabritious et al.33 used a genetic algorithm-based optimization method to derive improved model coefficients for some of the most common RANS turbulence models, including kε, kω, and Spalart–Allmaras. Janiga et al.34 used the PIV data at Re = 6500 to show that their large eddy simulation (LES) model predicted the temporally averaged flow velocities better than commonly used RANS models. Zmijanovic et al.35 showed that the downstream jet breakdown location for transitional flow through the nozzle was strongly affected by the solver numerics (grid size, temporal integration scheme, time step) and low-level inlet perturbations. Bhushan et al.,36 Delorme et al.,37 and Passerini et al.38 used the data to validate their hybrid RANS/LES, LES/immersed boundary method, and direct numerical simulation solvers, respectively.

Subsequent experimental studies have also been performed on the nozzle model to more fully characterize the flow fields for CFD validation. To resolve flow regions with large velocity ranges and gradients (such as the region downstream of the nozzle throat where a high-velocity jet is bounded by recirculation zones at the walls), Raben et al.3 used increased PIV image spatial resolution, ensemble correlation, dynamic range enhancement, and phase correlations to increase signal-to-noise ratios and measurement accuracy compared with the original multi-laboratory PIV measurements. As local velocity uncertainty was found to vary greatly and depend largely on local conditions such as particle seeding, velocity gradients, and particle displacements, suggestions for improving PIV measurement practices in medical device evaluations have also been proposed.3,24 Recently, high spatial resolution two-component laser Doppler velocimetry (LDV) measurements in the jet shear layer downstream of the nozzle throat at Re = 5000 showed that peak turbulent shear stresses of 1000–2000 Pa, which may impact predictive models of hemolysis, were an order of magnitude higher than the peak viscous shear stresses (<100 Pa).39

As CFD is often used in the design and evaluation of circulatory assist devices,1,2,4 the second benchmark model consisted of a centrifugal blood pump with a simple rotor and housing geometry. Pressure head measured across the pump matched well between the PIV and hemolysis tests, whereas some individual CFD predictions varied significantly from the experiments. Although the majority of the pressure head estimates from the CFD submissions were within the experimental bounds, this does not imply that these models are considered validated for the present blood pump. To use CFD to predict hemolysis in blood pumps, validation should be performed at two different levels. The first level of validation should focus on the fluid dynamics including pressure, velocity, and viscous and turbulent stresses. The second level should focus on validating predictions of the biological response (e.g., relative and absolute levels of hemolysis). The amount and rigor of validation required at each level is a function of two factors: 1) how influential the model is in the decision-making process, and 2) the consequence to the patient if the decision made based on the model results is wrong.14 Hence, as the influence of the computational model increases in the regulatory decision-making process, the rigor of validation required to establish its credibility will also increase accordingly. An important, and often overlooked, aspect for medical devices is to ensure that the geometry used in the CFD simulation is the same as the finished, fabricated device and not the nominal “as-designed” device. If the fabricated and design dimensions differ significantly in critical regions of the device (e.g., rotor tip gap of a ventricular assist device, radius of curvature on a sharp corner, surface roughness), this may impact predictions of the local flow fields, shear stresses, blood damage, and the overall performance of the device, regardless of whether the physics are accurately modeled.

This summary article mainly focused on one pump operating condition (6 L/min, 3500 rpm), one plane for the velocity field analysis (i.e., in the upper-blade plane that transected the cutwater and exit tube), and only the steady-state CFD submissions. Reports detailing further results of the CFD and PIV analyses throughout the entire pump, such as the shear stress fields, will follow. Within the body of the pump, which is dominated by rotational flow from the spinning rotor, velocity fields measured in three laboratories using PIV matched well to most of the steady-flow CFD simulations. The largest discrepancies between the CFD and PIV results occurred in the prediction of where the flow separates within the diffuser region. Particle image velocimetry experiments at high flow rates consistently showed a jet angled from the center of the exit volute toward the outer wall of the diffuser, whereas several CFD simulations predicted it on the inner wall or central core of the diffuser. Our subsequent studies will focus on analyzing how different simulation models and input parameters, such as turbulence model, mesh density, and solver type (steady versus unsteady), affect the accuracy of the CFD results in comparison to the interlaboratory PIV and hemolysis results.

The benchmark blood pump model met its design goal of producing more hemolysis than the clinical blood pump at each test condition. This demonstrates that the source of the hemolysis was primarily caused by the blood pump model and not the other components of the flow loops (e.g., backpressure clamp, reservoir, heat exchanger). However, the hemolysis results for the pump model were found to be limited for blood damage validation as the six pump test conditions did not create highly differentiable levels of hemolysis (i.e., generated hemolysis levels for conditions 2–5 were not significantly different from each other). Although the variability in the hemolysis results was high (coefficient of variation was 58–133% for the test conditions), it was comparable to the variability reported for in vitro testing of a clinical blood pump (18–94%).25 The hemolysis level increased significantly at test condition 6 compared with the other test conditions. In preliminary experiments using high-speed video recording of sodium iodide solution at 7 L/min (and 25°C), intermittent cavitation bubbles were visually observed at the cutwater lip. Further studies are needed to determine whether increased blood damage occurred because of infrequent cavitation or from increased flow-induced shear stresses at this test condition.

Notably, only 37% of all CFD submissions provided a hemolysis prediction (11 of 28 submissions for the nozzle, and 8 of 24 submissions for the pump). Possible reasons for the low response rate are that hemolysis solvers are not standard in commercial software and users have to implement the models themselves, or there may be a lack of confidence in the hemolysis models. Hariharan et al.15 recently identified some common user errors in CFD predictions of hemolysis (e.g., incorrect hemolysis units based on experimental data, improper mass-flow weighting). We also observed errors in the benchmark pump study, as three of the eight submitted absolute hemolysis data set predictions did not correspond to their calculated relative hemolysis (RHI) values. This again emphasizes the need for credible verification and validation of hemolysis predictions, both on a relative and absolute scale, to support CFD simulations. To help with the first step in this process, Hariharan et al.15 provided idealized flow-based benchmarks to verify the implementation of commonly cited power law-based hemolysis models in CFD.

A limitation to the current study is that the hemolysis results may only be applicable to the specific blood properties (i.e., porcine blood, 36% hematocrit, 25°C, ACDA anticoagulation, blood age up to 56 hours) and pump model used during testing in one laboratory. Taskin et al.16 reported that power law–based predictions of hemolysis using equation constants from experiments in other laboratories could not accurately reproduce absolute hemolysis levels in blood pump models in their laboratory. However, they did find that an Eulerian scalar transport model of hemolysis estimation could be used for making relative hemolysis comparisons.16 As many factors can affect blood damage test results (e.g., blood species, variability between donors, blood age, anticoagulation, hematocrit, platelet count, temperature, plasma protein and lipid content, glucose level, and pH),40 we advocate interlaboratory blood damage testing over a range of blood preparation conditions to generate robust data for validation on a relative and absolute scale.25 To this end, the FDA is currently working with device industry partners through the Medical Device Innovation Consortium (www.MDIC.org) to develop other benchmark test models and data sets for improving predictive models of blood damage (hemolysis and thrombosis).

In summary, as noted in the FDA-recognized ISO circulatory support device standard,7 the current recommended use of CFD is limited to evaluating relative changes during the device design stage rather than assessing absolute values of predicted quantities. To be further considered as a regulatory-grade tool, CFD simulations must undergo a systematic hierarchical validation of predictions of the flow field, shear stress distribution, and biological response in a device. Commonly used models of blood damage rely on calculations of shear stresses from local velocity gradients. If CFD cannot accurately predict flow patterns in a device, then it cannot provide reliable predictions of the local shear stress field and, consequently, mechanical blood damage.3,24 Our aim was to generate detailed hydrodynamic characterization and hemolysis data for flows through a nozzle and blood pump model to support needed improvements in CFD validation processes for medical devices. Besides the possibility of damage to red blood cells and platelets from high shear stresses, it is also critical to identify recirculation zones in medical devices as they can be the nexus for thrombus formation. Our two benchmark CFD challenges identified significant discrepancies between experimental and computational results that were most prominent in regions where the flow area expanded and flow separation and recirculation occurred. Clearly, more work is needed to identify computational modeling techniques which can better resolve transitional and separating flows, as these characteristics are likely to occur in blood-contacting medical devices. Through the generation of standards,14 FDA Guidance Documents,12 and publicly available experimental benchmark models and data sets, industry, academia, and regulatory bodies can continue to collaboratively develop CFD methodologies to improve the credibility of simulations, which will eventually impact the safety and effectiveness evaluations of medical devices.


Valuable assistance for this project was provided by members of the Technical Planning Committee, including the following: Dr. Sandy Stewart (FDA), Dr. Greg Burgreen (Mississippi State University), Dr. Eric Paterson (Virginia Tech University), and Dr. Matt Myers (FDA). Matthew Giarra (Rochester Institute of Technology) was the principal design engineer for the blood pump model, under the direction of SWD and in collaboration with RAM. Our thanks to FDA personnel Dr. Yao Huang for statistical support, and Dr. Tina Morrison for manuscript review.


1. Burgreen GW, Antaki JF, Wu ZJ, Holmes AJ. Computational fluid dynamics as a development tool for rotary blood pumps. Artif Organs. 2001.25: 336340.
2. Marsden AL, Bazilevs Y, Long CC, Behr M. Recent advances in computational methodology for simulation of mechanical circulatory assist devices. Wiley Interdiscip Rev Syst Biol Med. 2014.6: 16988.
3. Raben JS, Hariharan P, Robinson R, Malinauskas R, Vlachos PP. Time-resolved particle image velocimetry measurements with wall shear stress and uncertainty quantification for the FDA nozzle model. Cardiovasc Eng Technol. 2016.7: 722.
4. Fraser KH, Zhang T, Taskin ME, Griffith BP, Wu ZJ. A quantitative comparison of mechanical blood damage parameters in rotary ventricular assist devices: Shear stress, exposure time and hemolysis index. J Biomech Eng. 2012.134: 081002.
5. ISO 5840–2:2015. Cardiovascular implants – Cardiac valve prostheses – Part 2: Surgically implanted heart valve substitutes. 2015. Arlington, VA: International Organization for Standardization (ISO).
6. ISO 5840–3:2013. Cardiovascular implants – Cardiac valve prostheses – Part 3: Heart valve substitutes implanted by transcatheter techniques. 2013. Arlington, VA: International Organization for Standardization (ISO).
7. ISO 14708-5:2010. Implants for surgery – Active implantable medical devices – Part 5: Circulatory support devices. 2010. Arlington, VA: International Organization for Standardization (ISO).
8. Oberkampf WL, Trucano TG, Hirsch C. Verification, validation, and predictive capability in computational engineering and physics. Appl Mech Rev. 2004.57: 345384.
9. Stewart SFC, Paterson EG, Burgreen GW, et al. Assessment of CFD performance in simulations of an idealized medical device: Results of FDA’s first computational interlaboratory study. Cardiovasc Eng Technol. 2012.3: 139160.
10. Stewart SFC, Hariharan P, Paterson EG, et al. Results of FDA’s first interlaboratory computational study of a nozzle with a sudden contraction and conical diffuser. Cardiovasc Eng Technol. 2013.4: 374391.
11. Sotiropoulos F. Computational fluid dynamics for medical device design and evaluation: Are we there yet? Cardiovasc Eng Technol. 2012.3: 137138.
12. Guidance for Industry and Food and Drug Administration Staff—Reporting of Computational Modeling Studies in Medical Device Submissions. 2016. US: Silver Spring, Maryland.
13. ASME V&V 20–2009—Standard for Verification and Validation in Computational Modeling of Fluid Dynamics and Heat Transfer. 2009. New York, NY: American Society of Mechanical Engineers (ASME).
14. Draft Standard (under development)—ASME V&V 40: Verification and Validation in Computational Modeling of Medical DevicesNew York, NY: American Society of Mechanical Engineers (ASME).
15. Hariharan P, D’Souza G, Horner M, et al. Verification benchmarks to assess the implementation of computational fluid dynamics based hemolysis prediction models. J Biomech Eng. 2015.137: 094501.
16. Taskin ME, Fraser KH, Zhang T, Wu C, Griffith BP, Wu ZJ. Evaluation of Eulerian and Lagrangian models for hemolysis estimation. ASAIO J. 2012.58: 36372.
17. Zhang T, Taskin ME, Fang HB, et al. Study of flow-induced hemolysis using novel Couette-type blood-shearing devices. Artif Organs. 2011.35: 11806.
18. Chen Y, Sharp MK. A strain-based flow-induced hemolysis prediction model calibrated by in vitro erythrocyte deformation measurements. Artif Organs. 2011.35: 14556.
19. Arora D, Behr M, Pasquali M. A tensor-based measure for estimating blood damage. Artif Organs. 2004.28: 100215.
20. Morshed KN, Bark D Jr, Forleo M, Dasi LP. Theory to predict shear stress on cells in turbulent blood flow. PLoS One. 2014.9: 117.
21. Fogelson AL, Neeves KB. Fluid mechanics of blood clot formation. Annu Rev Fluid Mech. 2015.47: 377403.
22. Leiderman K, Fogelson A. An overview of mathematical modeling of thrombus formation under flow. Thromb Res. 2014.133 (suppl 1): S12S14.
23. Taylor JO, Meyer RS, Deutsch S, Manning KB. Development of a computational model for macroscopic predictions of device-induced thrombosis. Biomech Model Mechanobiol. 2016. DOI: 10.1007/s10237-016-0793-2..
24. Hariharan P, Giarra M, Reddy V, et al. Multilaboratory particle image velocimetry analysis of the FDA benchmark nozzle model to support validation of computational fluid dynamics simulations. J Biomech Eng. 2011.133: 041002.
25. Herbertson LH, Olia SE, Daly A, et al. Multilaboratory study of flow-induced hemolysis using the FDA benchmark nozzle model. Artif Organs. 2015.39: 23748.
26. Olia SE, Herbertson LH, Malinauskas RA, Kameneva MV. A reusable, compliant, small volume blood reservoir for in vitro hemolysis testing. Artif Organs. 2016. DOI: 10.1111/aor.12724.
27. Malinauskas RA. Plasma hemoglobin measurement techniques for the in vitro evaluation of blood damage caused by medical devices. Artif Organs. 1997.21: 12551267.
28. Grubbs FE. Procedures for detecting outlying observations in samples. Technometrics. 1969.11: 121.
29. Garon A, Farinas MI. Fast three-dimensional numerical hemolysis approximation. Artif Organs. 2004.28: 10161025.
30. U.S. Food and Drug Administration. Computational Fluid Dynamics: An FDA Critical Path Initiative (Project repository website https://fdacfd.nci.nih.gov). Accessed 8 Sept. 2016.
31. White AT, Chong CK. Rotational invariance in the three-dimensional lattice Boltzmann method is dependent on the choice of lattice. J Comput Phys. 2011.230: 63676378.
32. Trias M, Arbona A, Masso J, Minano B, Bona C. FDA’s nozzle numerical simulation challenge: Non-Newtonian fluid effects and blood damage. PLoS One. 2014.9: e92638.
33. Fabritius B. Application of genetic algorithms to problems in computational fluid dynamics. University of Exeter. 2014.
34. Janiga G. Large eddy simulation of the FDA benchmark nozzle for a Reynolds number of 6500. Comput Biol Med. 2014.47: 113119.
35. Zmijanovic V, Mendez S, Moureau V, Nicoud F. About the numerical robustness of biomedical benchmark cases: Interlaboratory FDA’s idealized medical device. Int J Numer Method Biomed Eng. 2016e02789.
36. Bhushan S, Walters DK, Burgreen GW. Laminar, turbulent, and transitional simulations in benchmark cases with cardiovascular device features. Cardiovasc Eng Technol. 2013.4: 408426.
37. Delorme YT, Anupindi K, Frankel SH. Large eddy simulation of FDA’s idealized medical device. Cardiovasc Eng Technol. 2013.4: 392407.
38. Passerini T, Quaini A, Villa U, Veneziani A, Canic S. Validation of an open source framework for the simulation of blood flow in rigid and deformable vessels. Int J Numer Method Biomed Eng. 2013.29: 1192213.
39. Taylor J, Good B, Paterno A, et al. Analysis of transitional and turbulent flow through the FDA benchmark nozzle model using laser Doppler velocimetry. Cardiovasc Eng Technol. 2016.7: 191209.
40. Mueller MR, Schima H, Engelhardt H, et al. In vitro hematological testing of rotary blood pumps: Remarks on standardization and data interpretation. Artif Organs. 1993.17: 103110.

computational fluid dynamics; validation; medical devices; particle image velocimetry; in vitro hemolysis testing

Supplemental Digital Content

Copyright © 2017 by the ASAIO