Secondary Logo

Journal Logo

Accuracy of Continuous Noninvasive Hemoglobin Monitoring

A Systematic Review and Meta-Analysis

Kim, Sang-Hyun, MD, PhD*; Lilot, Marc, MD*; Murphy, Linda Suk-Ling, MLIS; Sidhu, Kulraj S., MD*; Yu, Zhaoxia, PhD; Rinehart, Joseph, MD*; Cannesson, Maxime, MD, PhD

doi: 10.1213/ANE.0000000000000272
Technology, Computing, and Simulation: Research Report
Free
SDC

BACKGROUND: Noninvasive hemoglobin (Hb) monitoring devices are available in the clinical setting, but their accuracy and precision against central laboratory Hb measurements have not been evaluated in a systematic review and meta-analysis.

METHODS: We conducted a comprehensive search of the literature (2005 to August 2013) with PubMed, Web of Science and the Cochrane Library, reviewed references of retrieved articles, and contacted manufactures to identify studies assessing the accuracy of noninvasive Hb monitoring against central laboratory Hb measurements. Two independent reviewers assessed the quality of studies using recommendations for reporting guidelines and quality criteria for method comparison studies. Pooled mean difference and standard deviation (SD) (95% limits of agreement) across studies were calculated using the random-effects model. Heterogeneity was assessed using the I2 statistic.

RESULTS: A total of 32 studies (4425 subjects, median sample size of 44, ranged from 10 to 569 patients per study) were included in this meta-analysis. The overall pooled random-effects mean difference (noninvasive—central laboratory) and SD were 0.10 ± 1.37 g/dL (−2.59 to 2.80 g/dL, I2 = 95.9% for mean difference and 95.0% for SD). In subgroup analysis, pooled mean difference and SD were 0.39 ± 1.32 g/dL (−2.21 to 2.98 g/dL, I2 = 93.0%, 71.4%) in 13 studies conducted in the perioperative setting and were −0.51 ± 1.59 g/dL (−3.63 to 2.62 g/dL, I2 = 83.7%, 96.4%) in 5 studies performed in the intensive care unit setting.

CONCLUSIONS: Although the mean difference between noninvasive Hb and central laboratory measurements was small, the wide limits of agreement mean clinicians should be cautious when making clinical decisions based on these devices.

Published ahead of print June 10, 2014.

From the Departments of *Anesthesiology & Perioperative Care, Science Library Reference, and Statistics, University of California Irvine, Orange, California.

Sang-Hyun Kim, MD, PhD, is currently affiliated with Department of Anesthesiology and Pain Medicine, Soonchunhyang University, Soonchunhyang University Bucheon Hospital, Bucheon, Republic of Korea.

Marc Lilot, MD, is currently affiliated with Department of Anesthesiology and Critical Care, Louis Pradel Hospital, University Claude Bernard Lyon 1, Lyon, France.

Accepted for publication February 18, 2014.

Published ahead of print June 10, 2014.

Funding: Support was provided solely from the Department of Anesthesiology and Perioperative Care at the University of California Irvine, Irvine, CA (Departmental funding). Dr. Sang-Hyun Kim was supported by a grant from the Soonchunhyang University, Soonchunhyang University Bucheon Hospital, Bucheon, Republic of Korea.

Conflicts of Interest: See Disclosures at the end of the article.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s web site.

Reprints will not be available from the authors.

Address correspondence to Maxime Cannesson, MD, PhD, Department of Anesthesiology & Perioperative Care, University of California, Irvine, 333 City Boulevard West Side, Orange, CA 92868-3301. Address e-mail to mcanness@uci.edu.

Back to Top | Article Outline

Rationale

Accurate perioperative measurement of hemoglobin (Hb) concentration is of major importance because it can impact decision making related to blood transfusion, which is a major public health issue.1 Standard Hb measurements are invasive, time-consuming, and painful for patients because they require direct blood sampling. New technologies, however, enable Hb to be measured noninvasively and continuously from a finger probe, and devices based on these technologies are increasingly used in clinical care.2 Currently, 2 systems are available for the measurement of Hb: multiwavelength co-oximetry (Radical-7™ and Pronto-7™; Masimo Corp., Irvine, CA) and occlusion spectroscopy (NBM-200™, OrSense, Nes Ziona, Israel). The accuracy and precision of these devices have been assessed against central laboratory Hb measurements in clinical settings ranging from critical patients in operating rooms, critical care units, and emergency departments to healthy volunteers in blood donation settings.3–10 However, the results of these studies, which are commonly presented with mean difference and 95% limits of agreement (LOA) using Bland–Altman analysis, have been controversial and inconsistent.11,12

Back to Top | Article Outline

Objective

We conducted a systematic review and meta-analysis of studies comparing noninvasive Hb measurements with invasive central laboratory measurements in adult and pediatric patients in critical care settings (operating rooms, intensive care, and emergency departments), as well as in healthy subjects in volunteer settings. The principal measures were the accuracy (mean difference), precision (standard deviation [SD] of mean difference), and 95% LOA of noninvasive Hb measurements compared to invasive central laboratory measurements.

Back to Top | Article Outline

METHODS

This systematic review and meta-analysis was conducted and constructed following the guidelines set forth in Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Appendix 1, Supplemental Digital Content 1, http://links.lww.com/AA/A905).13

Back to Top | Article Outline

Eligibility Criteria

The following characteristics were defined in advance as eligibility criteria for the studies to be included in our systematic review and meta-analysis:

  1. Published studies comparing Hb measured using commercially available noninvasive Hb monitoring systems to that measured by central laboratory Hb measurements in adult and child populations excluding neonates.
  2. Studies presenting mean difference or SD of the mean difference (or 95% LOA) between noninvasive Hb monitoring systems and central laboratory Hb measurements.
Back to Top | Article Outline

Information Sources and Search

We conducted a comprehensive search of the medical literature assessing the accuracy of noninvasive Hb monitoring against central laboratory Hb measurements via PubMed, Web of Science, and the Cochrane Library. To establish the search strategy, we first identified the main key (SpHb, oximeter, co-oximeter, co-oximetry, spectrophotometric, spectrophotometry, occlusion spectroscopy, rad-57, rad-87, radical-7, Masimo, NBM-200, OrSense, measurement, monitor, determination, continuous, beat-to-beat, real-time, Hb, and noninvasive). We then performed the literature search and applied the search strategy in each of the selected databases with no language restrictions for studies published from 2005 to August 2013 (Masimo Corp. released the commercial version of SpHb in 2005 and OrSense in 2009). The full electronic PubMed search strategy is presented in Appendix 2, Supplemental Digital Content 2, http://links.lww.com/AA/A906. In addition to the database search, we contacted the manufacturers of clinically available monitors, Masimo Corp. and OrSense, for other studies and we hand-searched references in the studies included in the full-text retrieval for studies that had not initially been identified.

Back to Top | Article Outline

Study Selection

Two investigators (SHK and ML) initially screened potentially eligible studies first by title and abstract, eliminating obviously unrelated work. The remaining studies were then retrieved in full text. They assessed eligibility according to inclusion criteria. If the eligibility of the study remained unclear, a third investigator (MC) made the final decision.

Back to Top | Article Outline

Data Collection Process

SHK and ML performed data extraction independently. A specifically designed data extraction sheet was used to perform data extraction from the individual studies. All data were then transferred separately to a standard Excel spreadsheet. SHK and ML reviewed each other’s extractions for inconsistencies and if needed returned to the original work to validate the correct data.

The minimum published data required to perform meta-analysis were mean difference and SD. We extracted mean difference and SD of mean differences between noninvasive and the central laboratory Hb measurements from a single study. If a study presented only mean difference and 95% LOA, SD was calculated as (upper LOA − mean difference)/1.96. If a Bland–Altman plot was presented but if mean difference and SD were not reported or if only the mean difference without SD or 95% LOA were reported, we contacted the authors of the study to obtain mean difference and SD. If we could not receive data from authors, SD was calculated when 95% confidence interval (CI) of mean difference was reported as follows:

where n = number of paired measurement, SE = standard error.

We standardized mean difference in the current meta-analysis to mean noninvasive measurement minus invasive Hb measurement and transformed source data as needed for reporting in this form.

We also extracted demographic data (age, gender distribution, height, weight, or body mass index) and other study characteristics: country where the study was performed, source of funding (industry versus nonindustry), study setting (operating room, intensive care unit, emergency department, blood donation, outpatient clinic or inpatient ward, or study volunteers), name and software version of noninvasive Hb monitoring devices and sensor, name of central laboratory Hb analyzer, sampling site of blood, numbers of patients enrolled and included in the analysis, and total number of paired measurements.

Back to Top | Article Outline

Risk of Bias in Individual Studies

Because there are no specific guidelines for the quality assessment of individual studies in a meta-analysis for method comparison studies, we modified existing guidelines to create an appropriate quality assessment tool. First, we implemented items suggested by the Guidelines for Reporting Reliability and Agreement Studies14 and the main quality criteria for method comparison studies previously suggested.11 These were incorporated in the Quality Assessment of Diagnostic Accuracy Studies guidelines (QUADAS-2).15 We also modified signaling questions and 4 domains for the assessment of risk of bias (patient selection, index test, reference standard, as well as flow and timing) and 3 domains for the assessment of concerns related to applicability (patient selection, index test, and reference standard) from QUADAS-2 guidelines to fit our needs. This process is detailed in Appendix 3, Supplemental Digital Content 3, http://links.lww.com/AA/A907. Using the modified QUADAS-2 form, 2 reviewers (SHK, ML) performed independent quality assessments on each study included in the meta-analysis. Risk for each of the bias domains and each of the applicability domains is classified as low, high, or unclear. Disagreement between reviewers was resolved by discussion with a third investigator (MC).

Back to Top | Article Outline

Summary Measures

Principal summary measures of the current meta-analysis were mean difference (defined as noninvasive−invasive measurement), SD, and 95% LOA (calculated by mean difference ±1.96 SD of mean differences).

Back to Top | Article Outline

Synthesis of Results

We used random-effects models to summarize overall mean difference and SD as previously reported.16,17 We calculated I2 to describe heterogeneity of mean differences and SDs across studies. The I2 describes the percentage of variation across studies that is due to heterogeneity rather than chance in which values of 25%, 50%, and 75% are considered low, moderate, and high, respectively.18 If there were substantial heterogeneity (I2 > 50%), we performed sensitivity analysis and meta-regression according to plausible clinical scenarios. Forest plots are presented with individual and random-effects pooled estimates of mean difference and 95% LOA to visualize the data.

Back to Top | Article Outline

Risk of Bias Across Studies

To assess for publication bias, we created funnel plots for mean difference of Hb against standard error for each study. These funnel plots were assessed visually for symmetry. In the absence of bias, these plots would resemble a symmetrical inverted funnel. To formally test for asymmetry, we applied Egger regression tests on secondary funnel plots using a significance level of 0.1 because of the small sample size.19

Back to Top | Article Outline

Additional Analysis

Sensitivity Analysis and Subgroup Analysis

We conducted sensitivity and subgroup analyses to explore the causes of heterogeneity. Sensitivity analysis was performed by removing studies in which SD was calculated from SE and on the studies classified as low risk both in the risk of bias domains and applicability concerns domains. We performed subgroup analysis for studies with multiple measurements per subject versus single measurement per subject. Other subgroup analyses included funding source (department versus industry), country of study, type of device, sampling site for the central laboratory Hb measurement, study setting (operating room, intensive care unit, emergency department, blood donation, outpatient clinic or inpatient ward, and study volunteers), and devices (Radical-7, Pronto-7, and NBM-200).

Back to Top | Article Outline

Meta-Regression

We conducted a meta-regression analysis on publication year, country, setting, funding, sample size, device, reference, sampling site, and total number of paired measurement.

All the calculations and tests were conducted using Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA) and R.20I2 of mean difference and SD of mean difference were calculated using the following equation:

where Q is test statistic Q used by DerSimonian and Laird,17 and df is the degrees of freedom. We were able to calculate I2 for both mean difference and SD because the calculation of I2 only requires the point estimates from individual studies (see Study Limitations). Data are presented as mean ± SD or mean difference ± SDs (95% LOA).

Back to Top | Article Outline

RESULTS

Study Selection

Four hundred sixty-three articles were retrieved from the database searches and manufacturers after removing duplicates. Two investigators excluded 416 studies by title and abstract screening. The remaining 47 studies were retrieved as full-text articles and were assessed for eligibility.3–9,21–60 Fifteen articles24,26,35,37,39,41,45,51,54–60 were excluded after full-text review for failure to meet the inclusion criteria or insufficient data, despite efforts to contact authors for data (Appendix 4, Supplemental Digital Content 4, http://links.lww.com/AA/A908). Finally, 32 studies3–9,21–23,25,27–34,36,38,40,42–44,46–50,52,53 were included in the meta-analysis (Fig. 1).

Figure 1

Figure 1

Back to Top | Article Outline

Study Characteristics

A total of 4425 subjects were included in this meta-analysis. Characteristics of individual studies are presented in Table 1. Median sample size of the studies was 44 and ranged from 10 to 569. Across the 25 studies that reported age or gender, the mean age was 45 years and 48% were men. Of the 32 studies included in the meta-analysis, 17 studies3–6,9,21,27–29,34,38,42–44,47,48,50 were conducted in the perioperative and critical care settings (12 studies performed in the operating room,3,4,9,21,27–29,34,38,43,44,48 4 studies in the intensive care unit,5,6,42,47 and 1 study in both the operating room and the intensive care unit).50 Four studies were conducted in the emergency department,22,25,40,49 2 studies were conducted in blood donors,8,32 6 studies were conducted in outpatient clinic or inpatient wards,7,23,30,33,36,46 and 3 studies were conducted in volunteers during specific experiments (volume kinetics analysis31,53 and hemodilution).52 Twelve studies were conducted in the United States,3,4,23,25,29,36,43,44,46,48,50,52 8 studies were conducted in France,5,6,9,28,40,42,47,49 7 studies were conducted in Europe (excluding France),21,22,30–32,34,53 3 studies were conducted in Asia,8,27,38 and 2 studies were conducted in the Middle East.7,33 Twenty-two studies assessed the accuracy of Radical-7,3,4,6,9,21,22,25,27–29,31,34,38,42–44,47–50,52,53 4 studies assessed the accuracy of Pronto-7,23,30,33,36 2 studies assessed the accuracy of NBM-200,7,8 1 study assessed the accuracy of NBM-200MP,5 2 studies assessed both Pronto-7 and NBM-200,32 1 study assessed both Pronto-7 and NBM-200MP,40 and finally 1 study assessed the RAD-57.46 Nineteen studies measured central laboratory Hb from venous blood,4,5,7–9,21–23,25,30–34,36,40,42,49,53 12 from arterial blood,6,27–29,38,43,44,46–48,50,52 and 1 study measured central laboratory from both venous and arterial blood.3

Table 1

Table 1

Back to Top | Article Outline

Risk of Bias Within Studies

Results of quality assessment using the modified QUADAS-2 are presented in Appendix 5, Supplemental Digital Content 5, http://links.lww.com/AA/A909. The risk of bias was assessed as low in 16 studies for patient selection, in 28 studies for index test, in 29 studies for reference standard domain, and in 14 studies for the flow and timing domains. Concerns regarding 3 QUADAS-2 applicability domains were assessed as low risk in 32 studies for patient selection, in 28 studies for index test, and in 29 studies for reference standard domain. Interestingly, only 6 studies were rated as low risk both in risk of bias and applicability concerns according to our quality assessment tool incorporating recent recommendation and standards.5,9,28,36,42,44

Back to Top | Article Outline

Synthesis of Results

Overall Meta-Analysis

Individual, subtotal, and pooled random-effects mean difference and 95% LOA for the 32 included articles are shown in Figure 2. SD was calculated using the formula for conversion from 95% CI in 2 studies.47,50 The pooled mean difference and SD were 0.10 ± 1.37 g/dL (−2.59 to 2.80 g/dL). We found high level of heterogeneity for both mean difference (I2 = 95.9%) and SD (I2 = 95.0%).

Figure 2

Figure 2

Back to Top | Article Outline

Risk of Bias Across Studies

Funnel plot constructed for mean difference for Hb against SE appeared symmetrical, and Egger regression test for asymmetry showed nonsignificant P values (P = 0.861; Fig. 3).

Figure 3

Figure 3

Back to Top | Article Outline

Additional Analysis

Sensitivity Analysis and Subgroup Analysis

Pooled mean difference and SD in the sensitivity analysis after removing data from studies in which SDs were calculated from SE was not different from the overall meta-analysis. Sensitivity analysis conducted on the 6 studies classified as low risk both in the risk of bias domains and applicability concerns domains did not reduce heterogeneity across studies. The pooled mean difference and SD of these studies was 0.08 ± 1.46 g/dL (−2.79 to 2.94 g/dL). Subgroup analysis by number of paired measurements per patient, funding source, country of study, and sampling site of blood did not reduce heterogeneity compared with main analysis (Appendix 6, Supplemental Digital Content 6, http://links.lww.com/AA/A910).

Back to Top | Article Outline

Subgroup Analysis by Study Settings

Critical Care Setting (Perioperative, Intensive Care Unit, and Emergency Department)

The pooled mean difference and SD was 0.39 ± 1.32 g/dL (−2.21 to 2.98 g/dL) in the perioperative setting, −0.51 ± 1.59 g/dL (−3.63 to 2.62 g/dL) in the intensive care unit setting, and −0.39 ± 1.73 g/dL (−3.78 to 2.99 g/dL) in the emergency department setting. Heterogeneity was still high after sensitivity analysis according to study settings confined to critical care setting.

Back to Top | Article Outline

Outpatient Clinic or Inpatient Ward, Blood Donation Volunteers, and Study Volunteers

The pooled mean difference and SD was −0.14 ± 1.35 g/dL (−2.78 to 2.50 g/dL) for outpatient clinic and inpatient ward settings. In blood donation volunteers, mean difference and SD was −0.05 ± 1.04 g/dL (−2.10 to 1.99 g/dL). There was high heterogeneity across studies conducted in the outpatient and blood donation volunteers. The pooled mean difference and SD was −0.19 ± 1.01 g/dL (−2.16 to 1.78 g/dL) in studies using other volunteers; there was no heterogeneity in this subgroup (Appendix 6, Supplemental Digital Content 6, http://links.lww.com/AA/A910).

Back to Top | Article Outline

Subgroup Analysis by Device

Pooled mean difference and SD were similar among 3 devices. Heterogeneity for mean difference across studies was still substantial (Table 2).

Table 2

Table 2

Back to Top | Article Outline

Meta-Regression

There was significant residual heterogeneity after meta-regression analysis according to plausible explanatory scenarios. The publication year was associated with mean difference with P < 0.05 with meta-regression test, but there was still heterogeneity after accounting for the effect of this factor (Appendix 6, Supplemental Digital Content 6, http://links.lww.com/AA/A910).

Back to Top | Article Outline

DISCUSSION

Summary of Evidence

This meta-analysis of 32 studies including 4425 subjects and assessing accuracy and precision of noninvasive Hb measurement compared to central laboratory Hb measurements showed that the overall random-effects pooled mean difference and SD was 0.10 ± 1.37 g/dL (−2.59 to 2.80 g/dL). We found significant between-study heterogeneity for both mean differences and SD. In the perioperative and critical care setting, these were 0.39 ± 1.32 g/dL (−2.21 to 2.98 g/dL) and −0.51 ± 1.59 g/dL (−3.63 to 2.62 g/dL), respectively. Heterogeneity was still high even after sensitivity analysis including only studies in critical care settings. Among 32 studies meeting inclusion criteria in the current meta-analysis, only 6 were rated as low risk in both risk of bias and applicability concerns based on our quality assessment tool, which incorporated recent recommendations and standards for method comparison studies.

The goal of this meta-analysis was to assess the agreement between noninvasive Hb measurements and central laboratory Hb measurements. This topic has raised significant interest in the anesthesiology, critical care, and emergency settings during the past 5 years because of the impact such technologies may have on patient management and blood transfusion practices.12 Several small, single-center studies have been conducted on the topic, and our goal was to inform clinicians using this technology at the bedside about the accuracy and the precision of these devices. From our results, health care providers should still be cautious when making clinical decision about blood transfusion based on these technologies. For example, if Hb measured using a central laboratory analyzer was 10 g/dL, Hb measured using one of the noninvasive Hb measurement devices tested in this meta-analysis could range anywhere between 7.4 and 12.8 g/dL. In the operating room setting (where 13 of the 32 studies included in our meta-analysis were conducted), if Hb measured using a central laboratory analyzer was 10 g/dL, Hb measured using one of the noninvasive Hb measurement device could range anywhere between 7.8 and 13.0 g/dL. Practically speaking, based on the overall accuracy and precision of the current meta-analysis, we recommend measuring Hb invasively if the noninvasive Hb level is <8.69 g/dL. This implies that at least 2.5% of the laboratory Hb level will be <6.0 g/dL, which is the transfusion trigger in the current guidelines. Inversely, one can expect that 97.5% of the laboratory Hb level will lie above 10.0 g/dL when the noninvasive Hb level is higher than 12.69 g/dL. The decision to measure invasive Hb between these levels depends on the clinical situation.

Considering the importance of blood transfusion in the perioperative period and the commonly accepted transfusion triggers, the lack of accuracy and precision in the current generation of these devices may negatively affect clinicians’ decisions about their patients. It should be noted, however, that other point-of-care Hb measurement devices have similarly large error ranges, but this has not prevented their routine use in clinical care.23 For example, the HemoCue portable hemoglobinometer (HemoCue; HemoCue AB, Ängelholm, Sweden) when tested against central laboratory standards has been shown to have a mean difference and SD of −0.1 ± 1.6 g/dL, and other studies have shown similarly wide LOA.8,10,61,62 In a direct head-to-head clinical trial, the LOA of the HemoCue device were found to be narrower than the Radical-7, but a portion of patients had >1.0 g/dL and even a 1.5 g/dL error compared to central laboratory testing for both devices.21 This underlines the challenges our community has faced with in the past to evaluate new monitoring systems and define what makes a new monitor acceptable or not. As recently discussed by Riou,11 method comparison studies lack clear standards. This is reflected by the overall low quality of the studies included in our meta-analysis. We believe that clear standards for conducting method comparison studies would help our community to better evaluate new monitors while at the same time promoting innovation.

Moreover, clinical decisions are often based on serial Hb measurements, and the ultimate goal of these technologies would be to both decrease unnecessary transfusions and simultaneously prompt earlier intervention when truly needed.63 Because of this, some authors have suggested alternative methods for the evaluation of these noninvasive and continuous Hb measurement systems (error grid for instance).12 In our review, we focused only on the agreement between these new technologies and invasive central laboratory measurements because recommendations for blood transfusion are principally based on specific cutoff points.1 Thus, it is of major importance to be able to correctly classify patients as being above or below these thresholds, and this starts first with the evaluation of the agreement between a new technology and a “gold standard.”64,65

One of the findings from this meta-analysis is that 6 of the 32 studies included were rated as low risk both in risk of bias and applicability concerns according to the criteria we used in our revised QUADAS-2 assessment sheet (Appendix 5, Supplemental Digital Content 5, http://links.lww.com/AA/A909). This confirms what had already been suggested by other authors on the overall quality of conduct and report of method comparison studies.11 In the revised quality assessment sheet used in our study, we integrated recommendations from recent guidelines14 and opinion leaders, including suggestions from a recently published editorial on the topic11 describing the main quality criteria for the report of method comparison studies. Only 6 studies5,9,28,36,42,44 (19 %) included in this meta-analysis met all the criteria from our revised QUADAS-2. The inconsistency and the relatively poor quality of studies conducted on the topic suggest that guidelines and recommendations regarding how these studies should be conducted and reported would be beneficial to our community. Recommendations similar to those published for randomized control trials (Consolidated Standards of Reporting Trials)66 or diagnostic accuracy studies (Standards for Reporting of Diagnostic Accuracy)67 would be very useful.11

We found significant heterogeneity among studies included in our meta-analysis but were not able to identify causes even after performing a series of subgroup analyses, sensitivity analyses, and meta-regressions. Multiple factors may be responsible, and each factor may contribute to a small proportion of the total. Interestingly, several previously published meta-analyses of method comparison studies (arterial blood pressure measurements68 and temperature measurements)69,70 found similar heterogeneity and were similarly not able to identify causes, despite extensive statistical analysis. In the present meta-analysis, use of different devices, different populations, different central laboratory analyzers, subtle differences in arterial blood versus venous blood, as well as the quality of the studies included in the meta-analysis may have contributed to the overall variation. The precision of industry-supported studies was 0.24 ± 1.02.

Back to Top | Article Outline

Study Limitations

First, our meta-analysis does not assess the potential clinical utility of these noninvasive Hb measurement devices. While we understand that the potential impact of a new monitoring technology on clinical decision making, patient outcome, and/or patient safety encompasses far more than the simple assessment of accuracy and precision of these systems, we believe that Hb is such an important variable for patient management that strong recommendations about the way the accuracy and precision of these systems should be evaluated and reported should be published before any outcome study is conducted. In the future, we suggest that studies assessing the effectiveness of this technology using pragmatic approaches and comparative effectiveness research methodologies will help better understand the role of these devices in our daily practice.

Second, we studied a mixture of different devices for noninvasive and invasive Hb measurements in this meta-analysis. We included studies focusing on the evaluation of 3 different technologies (Radical-7, Pronto-7, NBM-200, and NBM-200MP), and we included studies evaluating the accuracy and precision of these devices against different central laboratory analyzers, using different blood origins (arterial and venous). Unfortunately, this is the way these studies have been conducted, which is part of the rationale for clear guidelines and standardization of method comparison studies.

Third, we did not present 95% CIs of mean difference and SD. We could not calculate the 95% CIs of the pooled SD because we did not have information regarding the accuracy of SD from individual studies. We have not found any publications that provide statistical methods to quantify the uncertainty of SD in meta-analysis. Consequently, we presented the results of the current meta-analysis with mean difference and 95% LOA as proposed previously.16,69,70 In addition, regarding the SD of the mean difference between 2 methods within a single study, some studies performed only 1 measurement per subject, whereas other studies performed multiple measurements per subject. If the current meta-analysis had complete data from all studies available, we would have had a multilevel (hierarchical) model and could have calculated the accuracy of SD. Without complete data from all the studies, given the information reported in at least some of those studies that had multiple measurements in each subject, we could not perform a multilevel hierarchical model. We performed subgroup analysis to account for the issue of multiple measurements. As demonstrated in subgroup analysis, the results are similar between studies with single and multiple measurements. Therefore, we assume that this issue does not confound the result substantially.

Finally, our research strategy was limited to PubMed, Web of Science, Cochrane Library, and to articles provided by manufacturers, and only included studies published in peer-reviewed journals. We limited the search to peer-reviewed publications to avoid low-quality manuscripts and because the impact of the inclusion of “gray literature” in meta-analyses is still unclear, especially for meta-analysis of method comparison studies. However, it is possible that different search strategies could result in different conclusions.

Back to Top | Article Outline

CONCLUSIONS

Our meta-analysis showed that the overall random-effects pooled mean difference and SD of noninvasive Hb measurements compared to invasive central laboratory measurements was 0.10 ± 1.37 g/dL (−2.59 to 2.80 g/dL). Although the mean difference between noninvasive Hb and central laboratory measurements were small, the wide LOA mean clinicians should be cautious when making clinical decisions based on these devices.

Back to Top | Article Outline

RECUSE NOTE

Dr. Maxime Cannesson is the Section Editor for Technology, Computing, and Simulation for the Journal. This manuscript was handled by Dr. Steven L. Shafer, Editor-in-Chief, and Dr. Cannesson was not involved in any way with the editorial process or decision.

Back to Top | Article Outline

DISCLOSURES

Name: Sang-Hyun Kim, MD, PhD.

Contribution: This author helped design and conduct the study, analyze the data, and write the manuscript.

Attestation: Sang-Hyun Kim has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Conflicts of Interest: The author has no conflicts of interest to declare.

Name: Marc Lilot, MD.

Contribution: This author helped design and conduct the study, analyze the data, and write the manuscript.

Attestation: Marc Lilot has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: The author has no conflicts of interest to declare.

Name: Linda Suk-Ling Murphy, MLIS.

Contribution: This author helped analyze the data and write the manuscript.

Attestation: Linda Suk-Ling Murphy has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: The author has no conflicts of interest to declare.

Name: Kulraj S. Sidhu, MD.

Contribution: This author helped conduct the study and analyze the data.

Attestation: Kulraj S. Sidhu has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: The author has no conflicts of interest to declare.

Name: Zhaoxia Yu, PhD.

Contribution: This author helped design the study, analyze the data, and write the manuscript.

Attestation: Zhaoxia Yu has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: The author has no conflicts of interest to declare.

Name: Joseph Rinehart, MD.

Contribution: This author helped analyze the data and write the manuscript.

Attestation: Joseph Rinehart has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: Joseph Rinehart is a speaker for Masimo Corp. Joseph Rinehart has equity interest in Sironis.

Name: Maxime Cannesson, MD, PhD.

Contribution: This author helped design and conduct the study, analyze the data, and write the manuscript.

Attestation: Maxime Cannesson has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Conflicts of Interest: Maxime Cannesson is a consultant for Edwards Lifesciences (Irvine, CA), Covidien (Boulder, CO), Masimo Corp. (Irvine, CA), Gauss Surgical (Palo Alto, CA), and Philips Medical System (Suresnes, France). Maxime Cannesson has equity position in Sironis.

Back to Top | Article Outline

REFERENCES

1. Goodnough LT, Shander A. Patient blood management. Anesthesiology. 2012;116:1367–76
2. Cannesson M, Talke P. Recent advances in pulse oximetry. F1000 Med Rep. 2009;1:66
3. Berkow L, Rotolo S, Mirski E. Continuous noninvasive hemoglobin monitoring during complex spine surgery. Anesth Analg. 2011;113:1396–402
4. Butwick A, Hilton G, Carvalho B. Non-invasive haemoglobin measurement in patients undergoing elective Caesarean section. Br J Anaesth. 2012;108:271–7
5. Coquin J, Bertarrex A, Dewitte A, Lefèvre L, Joannes-Boyau O, Fleureau C, Winnock S, Leuillet S, Janvier G, Ouattara A. Accuracy of determining hemoglobin level using occlusion spectroscopy in patients with severe gastrointestinal bleeding. Anesthesiology. 2013;118:640–8
6. Frasca D, Dahyot-Fizelier C, Catherine K, Levrat Q, Debaene B, Mimoz O. Accuracy of a continuous noninvasive hemoglobin monitor in intensive care unit patients. Crit Care Med. 2011;39:2277–82
7. Hadar E, Raban O, Bouganim T, Tenenbaum-Gavish K, Hod M. Precision and accuracy of noninvasive hemoglobin measurements during pregnancy. J Matern Fetal Neonatal Med. 2012;25:2503–6
8. Kim MJ, Park Q, Kim MH, Shin JW, Kim HO. Comparison of the accuracy of noninvasive hemoglobin sensor (NBM-200) and portable hemoglobinometer (HemoCue) with an automated hematology analyzer (LH500) in blood donor screening. Ann Lab Med. 2013;33:261–7
9. Lamhaut L, Apriotesei R, Combes X, Lejay M, Carli P, Vivien B. Comparison of the accuracy of noninvasive hemoglobin monitoring by spectrophotometry (SpHb) and HemoCue® with automated laboratory hemoglobin measurement. Anesthesiology. 2011;115:548–54
10. Naftalovich R, Naftalovich D. Error in noninvasive spectrophotometric measurement of blood hemoglobin concentration under conditions of blood loss. Med Hypotheses. 2011;77:665–7
11. Riou B. Continuous measurement of hemoglobin: methodological approach and lessons for the future. Anesthesiology. 2013;118:497–9
12. Rice MJ, Gravenstein N, Morey TE. Noninvasive hemoglobin monitoring: how accurate is enough? Anesth Analg. 2013;117:902–7
13. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151:W65–94
14. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96–106
15. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PMQUADAS-2 Group. . QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–36
16. Williamson PR, Lancaster GA, Craig JV, Smyth RL. Meta-analysis of method comparison studies. Stat Med. 2002;21:2013–25
17. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88
18. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–60
19. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34
20. R Development Core Team: R: A Language and Environment for Statistical Computing. 2010 Vienna, Austria R Foundation for Statistical Computing Available at: http://R-project.org
21. Skelton VA, Wijayasinghe N, Sharafudeen S, Sange A, Parry NS, Junghans C. Evaluation of point-of-care haemoglobin measuring devices: a comparison of Radical-7™ pulse co-oximetry, HemoCue(®) and laboratory haemoglobin measurements in obstetric patients*. Anaesthesia. 2013;68:40–5
22. Sjöstrand F, Rodhe P, Berglund E, Lundström N, Svensen C. The use of a noninvasive hemoglobin monitor for volume kinetic analysis in an emergency room setting. Anesth Analg. 2013;116:337–42
23. Shah N, Osea EA, Martinez GJ.. Accuracy of noninvasive hemoglobin and invasive point-of-care hemoglobin testing compared with a laboratory analyzer. Int J Lab Hematol. 2014;36:56–61
24. Pinto M, Barjas-Castro ML, Nascimento S, Falconi MA, Zulli R, Castro V. The new noninvasive occlusion spectroscopy hemoglobin measurement method: a reliable and easy anemia screening test for blood donors. Transfusion. 2013;53:766–9
25. Knutson T, Della-Giustina D, Tomich E, Wills B, Luerssen E, Reynolds P. Evaluation of a new nonnvasive device in determining hemoglobin levels in emergency department patients. West J Emerg Med. 2013;14:283–6
26. Joseph B, Hadjizacharia P, Aziz H, Snyder K, Wynne J, Kulvatunyou N, Tang A, O’Keeffe T, Latifi R, Friese R, Rhee P. Continuous noninvasive hemoglobin monitor from pulse ox: ready for prime time? World J Surg. 2013;37:525–9
27. Isosu T, Obara S, Hosono A, Ohashi S, Nakano Y, Imaizumi T, Mogami M, Murakawa M. Validation of continuous and noninvasive hemoglobin monitoring by pulse CO-oximetry in Japanese surgical patients. J Clin Monit Comput. 2013;27:55–60
28. Giraud B, Frasca D, Debaene B, Mimoz O. Comparison of haemoglobin measurement methods in the operating theatre. Br J Anaesth. 2013;111:946–54
29. Dewhirst E, Naguib A, Winch P, Rice J, Galantowicz M, McConnell P, Tobias JD.. Accuracy of Noninvasive and Continuous Hemoglobin Measurement by Pulse Co-Oximetry During Preoperative Phlebotomy. J Intensive Care Med. 2013
30. Bruells CS, Menon AK, Rossaint R, Goetzenich A, Czaplik M, Zoremba N, Autschbach R, Schaelte G. Accuracy of the Masimo Pronto-7® system in patients with left ventricular assist device-. J Cardiothorac Surg. 2013;8:159
31. Bergek C, Zdolsek JH, Hahn RG. Accuracy of noninvasive haemoglobin measurement by pulse oximetry depends on the type of infusion fluid. Eur J Anaesthesiol. 2013;30:73–9
32. Belardinelli A, Benni M, Tazzari PL, Pagliaro P. Noninvasive methods for haemoglobin screening in prospective blood donors. Vox Sang. 2013;105:116–20
33. Al-Khabori M, Al-Hashim A, Jabeen Z, Al-Farsi K, Al-Huneini M, Al-Riyami A, Al-Kemyani N, Daar S. Validation of a noninvasive pulse CO-oximetry-based hemoglobin estimation in patients with sickle cell disease. Int J Lab Hematol. 2013;35:e21–3
34. Vos JJ, Kalmar AF, Struys MM, Porte RJ, Wietasch JK, Scheeren TW, Hendriks HG. Accuracy of non-invasive measurement of haemoglobin concentration by pulse co-oximetry during steady-state and dynamic conditions in liver surgery. Br J Anaesth. 2012;109:522–8
35. Tekstra-Kay C, Schoeffel RE, Bolton D, Htun C, King GG. The use of non-invasive haemoglobin in the measurement of diffusion capacity. Respirology. 2012;17:6
36. Raikhel M. Accuracy of noninvasive and invasive point-of-care total blood hemoglobin measurement in an outpatient setting. Postgrad Med. 2012;124:250–5
37. Pyregov AV, Ovechkin A, Petrov SV.. [Noninvasive total hemoglobin monitoring based on multiwave spectrophotometry in obstetrics and gynecology]. Anesteziol Reanimatol. 2012:36–9
38. Park YH, Lee JH, Song HG, Byon HJ, Kim HS, Kim JT. The accuracy of noninvasive hemoglobin monitoring using the radical-7 pulse CO-Oximeter in children undergoing neurosurgery. Anesth Analg. 2012;115:1302–7
39. Miller RD, Ward TA, McCulloch CE, Cohen NH. Does a digital regional nerve block improve the accuracy of noninvasive hemoglobin monitoring? J Anesth. 2012;26:845–50
40. Gayat E, Aulagnier J, Matthieu E, Boisson M, Fischler M. Non-invasive measurement of hemoglobin: assessment of two different point-of-care technologies. PLoS One. 2012;7:e30065
41. Crowley C, Montenegro-Bethancourt G, Solomons NW, Schűmann K. Validity and correspondence of non-invasively determined hemoglobin concentrations by two trans-cutaneous digital measuring devices. Asia Pac J Clin Nutr. 2012;21:191–200
42. Coquin J, Dewitte A, Manach YL, Caujolle M, Joannes-Boyau O, Fleureau C, Janvier G, Ouattara A. Precision of noninvasive hemoglobin-level measurement by pulse co-oximetry in patients admitted to intensive care units for severe gastrointestinal bleeds. Crit Care Med. 2012;40:2576–82
43. Colquhoun DA, Forkin KT, Durieux ME, Thiele RH. Ability of the Masimo pulse CO-Oximeter to detect changes in hemoglobin. J Clin Monit Comput. 2012;26:69–73
44. Applegate RL 2nd, Barr SJ, Collier CE, Rook JL, Mangus DB, Allard MW. Evaluation of pulse cooximetry in patients undergoing abdominal or pelvic surgery. Anesthesiology. 2012;116:65–72
45. Alvarez RG, Motos AA, Duenas DS, Montero LG, Vecino JMC.. The accuracy of continuous noninvasive measurement of hemoglobin via pulse co-oximetry in patients undergoing knee arthroplasty. BJA. 2012;108:109–10
46. Ruppel GL, Wilson HA, Gall VK, Hempkens JA. Multi-wavelength pulse oximeter is not suitable for adjusting D(LCO) measurements. Respir Care. 2011;56:1115–21
47. Nguyen BV, Vincent JL, Nowak E, Coat M, Paleiron N, Gouny P, Ould-Ahmed M, Guillouet M, Arvieux CC, Gueret G. The accuracy of noninvasive hemoglobin measurement by multiwavelength pulse oximetry after cardiac surgery. Anesth Analg. 2011;113:1052–7
48. Miller RD, Ward TA, Shiboski SC, Cohen NH. A comparison of three methods of hemoglobin monitoring in patients undergoing spine surgery. Anesth Analg. 2011;112:858–63
49. Gayat E, Bodin A, Sportiello C, Boisson M, Dreyfus JF, Mathieu E, Fischler M. Performance evaluation of a noninvasive hemoglobin monitoring device. Ann Emerg Med. 2011;57:330–3
50. Causey MW, Miller S, Foster A, Beekley A, Zenger D, Martin M. Validation of noninvasive hemoglobin measurements using the Masimo Radical-7 SpHb Station. Am J Surg. 2011;201:592–8
51. Butwick AJ, Hilton G, Riley ET, Carvalho B. Non-invasive measurement of hemoglobin during cesarean hysterectomy: a case series. Int J Obstet Anesth. 2011;20:240–5
52. Macknet MR, Allard M, Applegate RL 2nd, Rook J. The accuracy of noninvasive and continuous total hemoglobin measurement by pulse CO-Oximetry in human subjects undergoing hemodilution. Anesth Analg. 2010;111:1424–6
53. Hahn RG, Li Y, Zdolsek J. Non-invasive monitoring of blood haemoglobin for analysis of fluid volume kinetics. Acta Anaesthesiol Scand. 2010;54:1233–40
54. Crowley C, Montenegro-Bethancourt G, Arriaga C, Solomons NW, Schumann K. Correspondence of hemoglobin values obtained by a noninvasive, cutaneous-contact method with values obtained by conventional methods from whole blood samples in a Guatemalan field setting. Food Nutr Bull. 2010;31:503–12
55. Sakudo A, Kato YH, Kuratsune H, Ikuta K. Non-invasive prediction of hematocrit levels by portable visible and near-infrared spectrophotometer. Clin Chim Acta. 2009;408:123–7
56. Bender JE, Shang AB, Moretti EW, Yu B, Richards LM, Ramanujam N. Noninvasive monitoring of tissue hemoglobin using UV-VIS diffuse reflectance spectroscopy: a pilot study. Opt Express. 2009;17:23396–409
57. Yoon G, Kim SJ, Jeon KJ. Robust design of finger probe in non-invasive total haemoglobin monitor. Med Biol Eng Comput. 2005;43:121–5
58. Petrova IY, Esenaliev RO, Petrov YY, Brecht HP, Svensen CH, Olsson J, Deyo DJ, Prough DS. Optoacoustic monitoring of blood hemoglobin concentration: a pilot clinical study. Opt Lett. 2005;30:1677–9
59. Noiri E, Kobayashi N, Takamura Y, Iijima T, Takagi T, Doi K, Nakao A, Yamamoto T, Takeda S, Fujita T. Pulse total-hemoglobinometer provides accurate noninvasive monitoring. Crit Care Med. 2005;33:2831–5
60. Kanashima H, Yamane T, Takubo T, Kamitani T, Hino M. Evaluation of noninvasive hemoglobin monitoring for hematological disorders. J Clin Lab Anal. 2005;19:1–5
61. Adam I, Ahmed S, Mahmoud MH, Yassin MI. Comparison of HemoCue® hemoglobin-meter and automated hematology analyzer in measurement of hemoglobin levels in pregnant women at Khartoum hospital, Sudan. Diagn Pathol. 2012;7:30
62. Mimoz O, Frasca D, Médard A, Soubiron L, Debaene B, Dahyot-Fizelier C. Reliability of the HemoCue® hemoglobinometer in critically ill patients: a prospective observational study. Minerva Anestesiol. 2011;77:979–85
63. O’Reilly M. Understanding noninvasive and continuous hemoglobin monitoring. Crit Care Med. 2013;41:e52
64. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10
65. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60
66. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang TCONSORT GROUP (Consolidated Standards of Reporting Trials). . The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001;134:663–94
67. Bossuyt PM, Reitsma JBStandards for Reporting of Diagnostic Accuracy. . The STARD initiative. Lancet. 2003;361:71
68. Cheng HM, Lang D, Tufanaru C, Pearson A.. Measurement accuracy of non-invasively obtained central blood pressure by applanation tonometry: a systematic review and meta-analysis. Int J Cardiol. 2013;167:1867–76
69. Craig JV, Lancaster GA, Williamson PR, Smyth RL. Temperature measured at the axilla compared with rectum in children and young people: systematic review. BMJ. 2000;320:1174–8
70. Craig JV, Lancaster GA, Taylor S, Williamson PR, Smyth RL. Infrared ear thermometry compared with rectal thermometry in children: a systematic review. Lancet. 2002;360:603–9

Supplemental Digital Content

Back to Top | Article Outline
© 2014 International Anesthesia Research Society